Interocular alignment, condition-adaptive illumination scheduling for single-panel near-eye displays

Abstract

Interocular alignment, conditionally adaptive illumination scheduling for single-panel near-eye displays is disclosed. A computer-implemented method for operating a near-eye display is provided, the near-eye display using a single image panel split into left-eye and right-eye regions, each region having an independently controllable illumination subsystem. The method includes: driving the near-eye display; sensing operating conditions of the display; determining, based at least in part on refresh status and operating conditions of each of the left-eye and right-eye regions, an illumination window for the region; and commanding the illumination subsystem to emit light during the determined illumination window, while coordinating the timing of the light emission for each region to satisfy interocular alignment and mitigate motion artifacts induced by the display. Various other aspects are also disclosed.

Description

[0001] Cross-reference and priority claims of related applications This application claims priority to U.S. Application No. 63 / 733,299, filed December 12, 2024, and U.S. Nonprovisional Patent Application No. 19 / 397,817, filed November 21, 2025, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0002] The disclosed subject matter provides a system and method for interocular alignment and conditional adaptive lighting scheduling in a single-panel near-eye display. Background Technology

[0003] Near-eye display systems, especially those employing single-image panels divided into left and right eye zones, have historically faced several technical challenges impacting visual fidelity and user comfort. Traditional designs often suffer from display ghosting, motion-to-photon latency, and brightness differences between the two zones (particularly in liquid crystal display (LCD) architectures). These limitations can produce noticeable artifacts, motion sickness, and reduced immersion, especially during rapid head movements or when viewing dynamic content. The slow response time of liquid crystals further exacerbates ghosting, while a lack of precise inter-eye alignment in the emission timing can introduce time discrepancies that negatively affect the user's visual experience. Summary of the Invention

[0004] This disclosure describes a computer-implemented method for operating a near-eye display using a single image panel divided into left-eye and right-eye regions, each with its own independently controllable illumination subsystem. The method includes: sensing operating conditions of the display; determining an illumination window for each region based on the refresh state of the respective region and the sensed operating conditions; and commanding the illumination subsystem to emit light during the calculated illumination window. The timing of light emission in each region is coordinated to maintain alignment between the eyes and reduce motion artifacts that may be caused by the display.

[0005] This disclosure also describes a system comprising at least one physical processor and a physical memory having computer-executable instructions. When executed, these instructions cause the processor to drive a near-eye display having a single image panel divided into a left-eye region and a right-eye region, each region having an independently controllable illumination subsystem. The system senses the operating conditions of the display, determines an illumination window for each region based on the refresh state and operating conditions, and commands the illumination subsystem to emit light during the determined illumination window. The system coordinates the emission timing of each region to maintain inter-eye alignment and mitigate motion artifacts.

[0006] Additionally, this disclosure describes a non-transitory computer-readable medium comprising a plurality of instructions, which, when executed by a processor, cause the processor to drive a near-eye display having a single image panel divided into left-eye and right-eye regions and an independently controllable illumination subsystem. The plurality of instructions enables the processor to sense the operating conditions of the display, determine an illumination window for each region based on the refresh state and operating conditions, and command the illumination subsystem to emit light during the calculated illumination window. The plurality of instructions also coordinate the emission timing of each region to maintain alignment between the eyes and reduce motion artifacts caused by the display. Attached Figure Description

[0007] The accompanying drawings illustrate several exemplary embodiments and are part of the specification. These drawings, together with the following description, illustrate and explain various principles of this disclosure.

[0008] Figure 1 This is a flowchart illustrating a method for operating a near-eye display using interocular alignment and conditional adaptive lighting scheduling.

[0009] Figure 2 It is a diagram depicting the timing and overlapping process of backlight unit emission relative to liquid crystal stabilization and frame transition.

[0010] Figure 3 It is a schematic diagram showing the brightness calculation process that uses duty cycle modulation and RGB intensity adjustment to optimize power and contrast.

[0011] Figure 4 It is a graph comparing the color accuracy of near-eye display systems with and without global offset adjustment.

[0012] Figure 5 This is an illustration of an example artificial reality system according to some embodiments of the present disclosure.

[0013] Figure 6 This is an illustration of an example artificial reality system with a handheld device according to some embodiments of the present disclosure.

[0014] Figure 7A These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0015] Figure 7B These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0016] Figure 8A These are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0017] Figure 8BThese are illustrations of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0018] Figure 9 This is an illustration of an example wrist-worn wearable device of an artificial reality system according to some embodiments of the present disclosure.

[0019] Figure 10 This is an illustration of an example wearable artificial reality system according to some embodiments of the present disclosure.

[0020] Figure 11 This is an illustration of an example augmented reality system according to some embodiments of the present disclosure.

[0021] Figure 12A This is an illustration of an example virtual reality system according to some embodiments of the present disclosure.

[0022] Figure 12B yes Figure 12A An illustration of another perspective of the virtual reality system shown.

[0023] Figure 13 It is a block diagram showing the system components of an example artificial reality system and a virtual reality system.

[0024] Throughout the accompanying drawings, the same reference numerals and descriptions indicate similar but not necessarily identical elements. While the exemplary embodiments described herein are readily adaptable and have alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the specific forms disclosed. Rather, this disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. Detailed Implementation

[0025] Near-eye display systems, especially those employing single-image panels divided into left and right eye zones, have historically faced several technical challenges impacting visual fidelity and user comfort. Traditional designs often suffer from display ghosting, motion-to-photon latency, and brightness differences between the two zones (particularly in liquid crystal display (LCD) architectures). These limitations can produce noticeable artifacts, motion sickness, and reduced immersion, especially during rapid head movements or when viewing dynamic content. The slow response time of liquid crystals further exacerbates ghosting, while a lack of precise inter-eye alignment in the emission timing can introduce time discrepancies that negatively affect the user's visual experience.

[0026] To address these issues, the disclosed subject provides a system and method for interocular alignment and conditionally adaptive illumination scheduling in a single-panel near-eye display. The method utilizes a single image panel divided into left-eye and right-eye regions, each with an independently controllable illumination subsystem. This system senses operating conditions such as panel temperature, refresh rate, and scan output position, and determines an illumination window for each region based on the refresh state and the sensed operating conditions. By commanding the illumination subsystem to emit light during the calculated illumination window and coordinating the emission timing of each region, the system maintains interocular alignment and reduces motion artifacts. This temperature-dependent backlight timing mechanism ensures that the liquid crystal has sufficient time to stabilize before illumination, thereby mitigating ghosting and reducing latency.

[0027] Furthermore, this theme incorporates a content-adaptive duty cycle algorithm to optimize power consumption and contrast. The system achieves energy savings by modulating the backlight duty cycle in response to content characteristics, while maintaining perceived brightness and enhancing contrast. The solution also includes global offset calibration to correct thermal color shift, ensuring consistent color accuracy across a variety of operating conditions. Through these integrated advancements, the described technology delivers improved visual fidelity, reduced latency, enhanced color accuracy, and increased user comfort in near-eye display applications.

[0028] Figure 1 An example method 100 for driving a near-eye display with inter-eye alignment and conditionally adaptive illumination scheduling is shown. The method is shown as a series of steps (110, 120, 130, and 140) that can be performed by one or more physical processors in electronic communication with a single image panel and an independently controllable illumination subsystem. Although Figure 1 The steps are presented in a specific order, but these steps can be executed in different orders, simultaneously, iteratively, or steps can be omitted or added, because the method is not limited to any fixed order.

[0029] A "near-eye display" is a display that presents an image within a short optical path distance of the user's eye (e.g., a head-mounted display, smart glasses, or other artificial reality glasses). An "image panel" is any display panel configured to render pixelated images, including but not limited to liquid crystal displays (LCDs), liquid crystal-on-silicon (LCOS) panels, micro-LED panels, organic light-emitting diode (OLED) displays, or waveguide-coupled projection displays. An "illumination subsystem" is a light-emitting component or assembly used to illuminate the image panel or otherwise provide controlled light emission, such as a backlight unit (BLU), a segmented strobe emitter, a strobe-enabled global shutter, a microprojector light source, or other optical illuminators. "Independently controllable" means that each illumination subsystem can be turned on, off, modulated, phase-shifted, or its duty cycle controlled without imposing the same behavior on another illumination subsystem.

[0030] Step 110 involves driving a near-eye display, the near-eye display including a single image panel divided into left-eye and right-eye regions and corresponding independently controllable illumination subsystems. In one embodiment, the single LCD panel is mechanically and electrically divided such that pixels in a first region correspond to the left-eye image and pixels in a second region correspond to the right-eye image. In this embodiment, the left BLU is optically coupled to the left region, the right BLU is optically coupled to the right region, and each BLU can be individually gated. In another embodiment, the illumination subsystem is segmented across the single panel, wherein each segment is assigned to the left-eye and right-eye regions and is driven independently to simulate two backlights without requiring separate BLU modules. In some embodiments, the illumination subsystem is an independently controllable backlight disposed behind the image panel.

[0031] In other embodiments, the illumination subsystem has segmented illumination sources arranged as partitions that can be independently gated for left and right eye regions, including partitioned arrays of LEDs or microLEDs that form a light field aligned with the corresponding region. Either approach allows the system to limit illumination to pixels in a given eye region at selected times.

[0032] A single image panel can be rotated relative to the optical axis of the lens, such that the raster scan of the panel is performed in opposite directions for the two eye regions. This rotation causes the refresh state of the left eye region to advance from left to right, for example, while that of the right eye region advances from right to left, thus producing the disclosed scheduling compensation for temporal asymmetry. This rotational configuration is very useful in head-mounted systems where mechanical packaging determines the orientation of the panel, and software adjusts the inter-eye alignment according to this orientation.

[0033] Driving a near-eye display includes providing pixel data, synchronizing frame and line timing, calibrating gamma and color, and commanding the illumination subsystem with control signals for each region. In one embodiment, the processor writes left-eye pixel data to the left-eye region and right-eye pixel data to the right-eye region, while managing the Mobile Industry Processor Interface-Display Serial Interface (MIPI-DSI) link timing to ensure support for the line scan interval of the scheduled illumination window. In another embodiment, the compositor or display processing unit performs chromatic aberration correction (CAC) on the rendered image before it is written to the panel, and the scheduling takes into account CAC pipeline latency.

[0034] This system enables brightness difference calibration between regions. In one embodiment, the processor measures or estimates the light output from each illumination subsystem and adjusts the left BLU current relative to the right BLU current to equalize the perceived brightness for the eye. Calibration can be static (applied during manufacturing) or dynamic (applied during use based on sensor readings or user feedback) and can be used in conjunction with scheduling described in subsequent steps.

[0035] Step 120 involves sensing the operating conditions of the near-eye display. As used herein, "operating conditions" are any state variables that affect panel behavior, illumination behavior, or system timing, including temperature, refresh rate, power supply voltage, illumination subsystem temperature, ambient temperature, content categorization, panel age, motion state, or head posture. Operating conditions can be at least temperature, such as panel temperature measured by a thermistor attached to the panel or by a panel temperature sensor embedded in the display stack.

[0036] In some embodiments, operating conditions include one or more of the following: ambient temperature measured by a housing sensor; panel temperature measured by a sensor thermally coupled to the panel; current refresh rate (e.g., 90 Hz vs. 120 Hz); power supply voltage delivered to the BLU; illumination subsystem temperature measured by a sensor on the BLU board; or panel age inferred from operating time. Each of these factors affects liquid crystal response, LED efficiency, and timing stability, thus requiring scheduling adjustments.

[0037] Sensing may include reading the motion state of a near-eye display or the user's head pose. In one embodiment, an inertial measurement unit (IMU) provides gyroscope and accelerometer data revealing head rotation speed and acceleration, and the system adjusts the emission timing of each region to better align with predicted head motion and reduce motion-to-photon latency. In another embodiment, an inside-out camera tracks the head pose and provides pose updates to a scheduler, which may slightly advance or delay the window to minimize perception lag.

[0038] Panel refresh information can also be considered as operational condition input. For example, the system can read row or row indices from the timing controller, detect frame boundary signals, or determine area update completion indicators exposed by the display driver. These signals (discussed further in step 130) inform the area that it has been fully written to account for the precise moment of illumination and are examples of sensing the “refresh state.”

[0039] Multiple sensors can be used for temperature sensing. In one embodiment, a panel sensor reports a temperature that tracks the liquid crystal viscosity, while a BLU sensor reports the LED junction temperature that affects luminous flux. The system can combine these readings to infer a composite “temperature condition” for scheduling. In another embodiment, a weighted average is used to fuse the ambient temperature with the panel temperature to increase stability across environments.

[0040] Power supply voltage sensing can help manage duty cycle and brightness uniformity. In one embodiment, the power management integrated circuit (PMIC) reports the BLU rail voltage, and when a voltage drop is detected, the scheduler reduces the duty cycle in non-critical regions to maintain stability and avoid flicker. As the liquid crystal and LED degrade, panel age can be tracked to adjust timing margins; for example, after a threshold number of operating times, the settling interval can be increased slightly to maintain an artifact-free presentation.

[0041] Step 130 involves determining the illumination window for each region, at least in part, based on the region's refresh state and operating conditions. As used herein, "refresh state" is a state variable indicating the progress of image data being written to the region, including but not limited to scan output position, row or row index, region update completion indicator, or frame boundary indicator. In one embodiment, the refresh state is the last row written to the region, and given the liquid crystal response at the current temperature, the scheduler calculates when that row and subsequent rows will stabilize.

[0042] An “illumination window” is typically the time interval during which the illumination subsystem emits light to render an image. The start and end of the window can be calculated to occur after the pixels have fully stabilized and before the start of the next frame to avoid blurring. In one embodiment, the window width is proportional to the duty cycle, and the window position is arranged after the settling time obtained from the response model.

[0043] For each region, the system can model the response characteristics of the image panel as a function of operating conditions and determine the illumination window based at least in part on the settling time obtained from the model. In one embodiment, the response model maps temperature to the liquid crystal rise and decay times of a typical gray-to-gray transition, and the scheduler selects a window start time equal to the refresh completion time plus the modeled rise time of the main gray level of the frame. In another embodiment, the response model incorporates voltage and aging to adjust the settling curve.

[0044] The refresh status can be affected by the rotation of the panel relative to the optical axis. When rotating a single panel, the left-eye area can scan in one direction, while the right-eye area scans in the opposite direction. The scheduler addresses this by calculating the different refresh completion times for each area, then determining the windows that are aligned across both eyes and arranged after the specific areas have stabilized sufficiently.

[0045] Operating conditions other than temperature can affect window placement. For example, at higher refresh rates (e.g., 120 Hz), line times are shorter, so the scheduler can reduce the duty cycle or move windows earlier to fit the frame budget while still meeting stability standards. When the power supply voltage drops, the scheduler can shorten the window duration to prevent power loss while maintaining the minimum expected brightness.

[0046] In some embodiments, the illumination windows overlap in time. This overlap can facilitate inter-eye alignment when the refresh completion times are slightly staggered. For example, the left window may be slightly later than the stabilization start time of the left eye region, and the right window may be slightly later than the stabilization start time of the right eye region, and the scheduler ensures that their overlap is within the target alignment tolerance, so that both eyes perceive the presentation as almost simultaneous.

[0047] Content characteristics may affect window duration. The scheduler can adaptively modulate the illumination duty cycle within at least one illumination window and, in response to duty cycle modulation, compensate for perceived brightness by adjusting effective brightness via pixel driving or illumination amplitude. In one embodiment, a darker scene triggers a reduced duty cycle to reduce image persistence, while the compositor increases RGB pixel values ​​to maintain perceived brightness, thereby improving contrast and reducing BLU power.

[0048] Head motion state can also be adjusted for window determination. When the IMU reports a rapid yaw rotation, the scheduler can advance the window slightly by two windows to reduce motion-to-photon delay while maintaining inter-eye alignment, thereby reducing perceptual lag during rapid head movements. Alternatively, during static scenes, the scheduler can extend the window duration to provide higher brightness while still avoiding artifacts. In many embodiments, window determination includes enforcing an inter-eye temporal parallax constraint. This constraint can be represented as a maximum permissible temporal offset between illumination windows, for example, an offset not exceeding a perceptual threshold calibrated through testing. The scheduler selects a window position that satisfies this constraint while maintaining settling-based artifact reduction, thus reconciling the two objectives.

[0049] The scheduler can adapt to pipeline changes. Determining the window can include adjusting the data link rate (e.g., increasing the MIPI rate) or pipeline scheduling to align the refresh state with coordinated luminescence timing. In one embodiment, the system increases the MIPI-DSI rate during high-motion scenes to reduce line time and increase the margin available for window layout without increasing latency.

[0050] Color correction (CAC) latency can be taken into account. Performing CAC on the system-on-chip can increase processing latency. The scheduler can address this by making the refresh state based on the CAC-written timing of the panel rather than the CAC-rendered timing, thereby ensuring that the window is aligned with the actual panel update completion time.

[0051] Step 140 involves: commanding the illumination subsystem to emit light during the defined illumination window, while coordinating the timing of light emission in each region to satisfy inter-eye alignment and mitigate motion artifacts caused by the display. As used herein, “coordinating” includes issuing control signals such that the left and right illumination subsystems emit light according to their respective windows, and that the temporal relationship between the windows satisfies inter-eye alignment constraints.

[0052] In one embodiment, the processor drives general-purpose input / output (GPIO) pins or dedicated BLU drivers with precise timing markers to gating each backlight at the start of a computed window and maintain the window duration. This control can apply pulse-width modulation to the duty cycle setting and current settings to the amplitude and phase alignment signals to obtain target-to-eye timing. In another embodiment, this control is executed by an illumination controller that receives the scheduled window as timestamps and autonomously performs gating.

[0053] Mitigating motion artifacts caused by displays includes suppressing ghosting and smearing. In LCD embodiments, illumination is commanded after a stabilization interval derived from the temperature-dependent liquid crystal response, ensuring that most pixels have reached the target state, and the gating display does not show intermediate transition states. In light-emitting panel embodiments, coordination can reduce perceived jitter even with different scanning modes by ensuring that both eyes perceive frames at the same time. Coordination can include overlapping or interleaving strategies. For example, when the left eye region stabilizes slightly earlier than the right eye region, the scheduler can start the left window at its earliest acceptable time and the right window at its earliest time, ensuring that the overlap duration satisfies the interocular temporal parallax constraint. When overlapping is not feasible due to frame budget constraints, windows can be closely interleaved within limitations to produce near-simultaneous perception.

[0054] In some embodiments, the system reduces motion-to-photon latency by advancing at least one illumination window while maintaining inter-eye alignment. For example, during rapid head rotation, advancing both windows by a small amount relative to a typical layout can reduce perceived hysteresis, provided that this advance does not violate stability criteria. This advance amount can be calculated using motion prediction and rendered pose estimation based on IMU data.

[0055] This coordination can also incorporate calibration based on brightness differences between regions. In one embodiment, commanding emission includes: setting the left and right BLU currents differently to equalize perceived brightness while maintaining the scheduled window timing. Calibration can be based on measurements taken during manufacturing or operation using optical sensors or user feedback mechanisms.

[0056] Global color and gamut adjustments can run in conjunction with scheduling. In one embodiment, a global white point offset is applied to correct thermally induced color casts. This color correction is performed in the pixel pipeline and does not alter window timing, but the scheduler can monitor the processing latency of color correction and include it in the determination of the refresh state, thereby ensuring that coordination remains accurate.

[0057] Power optimization can be integrated with window commands. Content-adaptive duty control (CADC) can reduce the BLU duty cycle in low-to-medium grayscale areas to save power while increasing RGB values ​​or illumination amplitude to maintain brightness. The scheduler's window duration naturally reflects the duty cycle variation, and coordination maintains inter-eye alignment regardless of the duty cycle setting. Experiments involving gaming and productivity scenarios have shown a reduction in BLU power of up to approximately 0.46 watts (W), and the method adapts to this saving without sacrificing alignment.

[0058] In some embodiments, the system measures the results and feeds them back to the scheduler. For example, sensors indicate whether frame boundaries have drifted due to temporal variations caused by temperature, and the scheduler adjusts the window layout accordingly. The coordination layer can also record whether interocular temporal parallax constraints are always met and tune parameters to maintain alignment across device aging, voltage variations, and environmental changes.

[0059] This method supports alternative display technologies. In an LCOS embodiment employing projector illumination, the illumination subsystem comprises a projector light source and shutter mechanism, and schedules and controls projector pulses for the left and right eye regions. In a micro-LED embodiment, the illumination subsystem may be an on-board light-emitting driver with gating capabilities, and schedules and coordinates segment gating for each region. In each case, the timing of each region is coordinated for alignment and artifact mitigation, and the window is determined using a technically appropriate response model.

[0060] This method can be implemented across various devices and device types. While the algorithm has been explained for single-panel LCD headsets, many features can be generalized to other applications, including displays that support adjustable illumination. Important developments include sensing operating conditions, determining windows for each region based on refresh status and operating conditions, and coordinating illumination to achieve inter-eye alignment, while mitigating motion artifacts on the single panel using independent illumination for each eye.

[0061] "Perceived brightness" broadly refers to the brightness perceived by a typical user under typical viewing conditions, and can be maintained by adjusting pixel drives (e.g., RGB intensity values) or illumination amplitude (e.g., BLU current) or both. "Duty cycle" broadly refers to a portion of a frame during which the illumination subsystem emits light, and can be modulated to adjust image persistence, power, and contrast. "Motion-to-photon delay" is the time elapsed between user movement and the display of a corresponding image update to the user; motion-to-photon delay can be reduced by scheduling the window closer to the end of the refresh cycle, while taking into account stability and alignment constraints.

[0062] "Interocular temporal parallax constraint" broadly refers to the constraint limiting the relative temporal difference between the illumination of the left and right eye regions, ensuring that the perceived temporal alignment remains within a threshold determined by human perception experiments or device specifications. "Stability interval" is the time margin by which a panel or pixel reaches a stable state after a refresh; this interval can be derived from modeled responses or empirical calibration and can vary with operating conditions.

[0063] "Scan Output Position" and "Row or Row Index" broadly refer to the raster coordinates indicating the progress of data writing across the panel; "Region Update Complete Indicator" is a signal indicating that the left or right eye region has been completely written to a frame; "Frame Boundary Indicator" is a signal marking the end or beginning of a frame. "Data Link Rate" broadly refers to the transmission rate on the display interface (e.g., MIPI-DSI) and can be adjusted to align with refresh timing; "Pipeline Scheduling" refers to the order and timing of rendering, image processing (including CAC), and panel write operations.

[0064] "Chronic Aberration Correction" (CAC) is an image processing operation that compensates for optical dispersion in the lens by spatially remapping pixels or causing pixel color shifts to correct perceived fringes. Performing CAC may introduce additional processing latency, which the scheduler takes into account when calculating refresh state timing. "Head Pose" refers to the orientation and position of the user's head; sensing head pose can be used to predict the desired rendering time and fine-tune the window layout.

[0065] While the preceding paragraphs have described specific embodiments and examples, other variations may also be used. The scheduler may be adaptive or rule-based, may incorporate machine learning models trained for device behavior across temperature ranges, and may expose configuration parameters to developers or users. Coordination may be performed in firmware, driver software, or dedicated hardware modules and may be updated post-manufacturing via software updates.

[0066] In summary, step 110 establishes a single-panel, dual-lighting architecture and a rendering driver; step 120 senses one or more operating conditions, including temperature and motion; step 130 uses the refresh state and the sensed operating conditions to calculate the lighting window for each region, thereby enforcing inter-eye constraints and stabilization criteria; and step 140 commands illumination while coordinating the timing of each region to maintain inter-eye alignment and mitigate motion artifacts caused by the display, optionally integrating duty cycle modulation, brightness compensation, latency reduction, data link adjustment, and color correction.

[0067] Figure 2 A diagram illustrating the timing and overlap of backlight unit emission relative to liquid crystal stabilization and frame transition in a near-eye display system is shown. The diagram includes several components that interact to coordinate emission timing and mitigate motion artifacts caused by the display.

[0068] Frame N 202 represents a time interval corresponding to the display refresh cycle, serving as a time reference for subsequent operations such as scan output, stabilization, and illumination for the left and right eye regions. Right Activity 206 indicates the period for writing image data to the right eye region, thereby initiating the liquid crystal transition in that region. Right Stabilization 208 indicates the interval during which liquid crystal pixels in the right eye region are allowed to complete their transition to the target state, where the timing is affected by response characteristics and panel temperature. Maximum Overlap 210 specifies the maximum permissible time overlap between the illumination windows of the left BLU 220 and the right BLU 216, constrained by inter-eye alignment requirements. Frame N+1 204 marks the start of the next refresh cycle, thereby initiating a new scan output and illumination process.

[0069] Left stabilization 214 defines the stabilization period for the liquid crystal pixels in the left eye region, ensuring the transition is completed before illumination. Right BLU 216 corresponds to the activation of the right backlight unit, is timed to follow the completion of right stabilization 208, and is independently controlled to adapt to operating conditions. BLU offset 218 represents the time offset between the activation of right BLU 216 and the activation of left BLU 220, and is calibrated to resolve differences between scan output time and stabilization time. Left BLU 220 indicates the activation of the left backlight unit, synchronized with the completion of left stabilization 214, and is independently controlled to achieve precise timing. Left activity 212 marks the period during which image data is written to the left eye region, which occurs after right activity 206 due to the sequential scan output in the rotating single-panel architecture. Each component contributes to coordinated emission timing, alignment between eye regions, and reduction of motion artifacts in the display system.

[0070] Figure 3 This diagram illustrates the brightness calculation process for power and contrast optimization in a near-eye display system using duty cycle modulation and RGB intensity adjustment. Brightness calculation formula 300 defines the relationship between duty cycle and RGB intensity, where brightness is determined by multiplying the duty cycle and RGB intensity. Duty cycle representation 310 shows a configuration with a 10% duty cycle and 50% RGB intensity, demonstrating the balance between backlight activation and pixel drive values. Visual representation 312 depicts the image quality obtained when the duty cycle is set to 10% and the RGB intensity is set to 50%, maintaining perceived brightness through compensatory adjustments. Duty cycle representation 320 presents a scene with a 5% duty cycle and 100% RGB intensity, showing a further reduction in backlight usage while increasing pixel drive to maintain brightness. Visual representation 324 shows the image quality when the duty cycle is reduced to 5% and the RGB intensity is set to 100%, demonstrating that perceived brightness is maintained despite lower backlight activation.

[0071] Figure 4The graph 400 compares the color accuracy of a near-eye display system under two calibration conditions: with global offset adjustment and without global offset adjustment. The system comprises a single image panel divided into left-eye and right-eye regions and a corresponding independently controllable illumination subsystem. The system is configured to sense the operating conditions of the near-eye display, determine an illumination window for each region based at least in part on the refresh state and operating conditions of that region, and command the illumination subsystem to emit light during the determined illumination window, while coordinating the emission timing of each region to satisfy inter-eye alignment and mitigate display-induced motion artifacts. In some aspects, the operating conditions include at least temperature and may include ambient temperature, panel temperature, refresh rate, power supply voltage, illumination subsystem temperature, or panel age. The system may also model the response characteristics of the image panel as a function of the operating conditions for each of the left-eye and right-eye regions, and determine the illumination window based at least in part on the settling time obtained from the model.

[0072] exist Figure 4 In the context of [the previous sentence], graph 400 illustrates the impact of global offset calibration on color accuracy by plotting the chromaticity coordinates of grayscale level 127 in two cases. The chromaticity coordinates are displayed in a two-dimensional color space, with the x-axis corresponding to the u′ chromaticity value and the y-axis corresponding to the v′ chromaticity value. The system can compensate for perceived brightness by adjusting the effective brightness via pixel driving or illumination amplitude in response to duty cycle modulation. For example, the system can implement a global offset to correct thermally induced color shift, thereby improving color accuracy across varying operating conditions. Global offset calibration is applied to the pixel pipeline and does not alter window timing; however, the scheduler can monitor the processing latency of color correction and include this latency in the refresh state determination, thus ensuring that coordination remains accurate.

[0073] This graph includes two sets of data points: "+" (plus) markers represent chromaticity coordinates when global offset adjustment is applied, while "-" (minus) markers represent coordinates without global offset adjustment. Overlapping dashed circles indicate tolerance areas for acceptable color accuracy, providing a visual reference for evaluating the effectiveness of the calibration. The "+" markers cluster more closely within the tolerance areas, indicating that global offset calibration can correct thermally induced color shifts and maintain consistent color accuracy. This is an example of how the system compensates for operational variations (e.g., temperature changes) by adjusting pixel drive or illumination amplitude to maintain perceived brightness and color fidelity.

[0074] In the comparative embodiment, a single-panel system that simply gates two per-eye illumination subsystems at exactly the same time, regardless of the refresh state of each region, can introduce significant inter-eye time delay imbalances and perceptual differences. When one region has just finished scanning and output, while another region is early in its scan, and the flash forces one eye to perceive pixels closer to its stable target, while the other eye perceives pixels in the intermediate transition, this can manifest as time jitter, binocular discomfort, and motion sickness. The disclosed scheduler avoids this failure mode by determining the window for each region from the region-specific refresh state and sensed conditions, and by coordinating these windows to satisfy inter-eye alignment constraints while still adhering to the stability criteria for each region.

[0075] In another comparative benchmark, a single backlight shared across two regions can only be gated after the latter region has completed its refresh. While this approach reduces ghosting compared to simultaneous dual-flash, it imposes an avoidable latency penalty on the earlier region and asymmetrically increases motion-to-photon latency across the eye. The single-backlight scheme also complicates brightness equalization and power management, as the entire panel must be driven to satisfy the slowest settling path. The disclosed dual-illumination architecture maintains low latency and artifact suppression by allowing each region to emit immediately after its modeled settling interval is satisfied, while conforming to inter-eye alignment. More details on Content Adaptive Duty Cycle Control (CADC) are now provided. CADC can operate as a temporal dimming mechanism that reduces the illumination duty cycle in response to scene brightness, contrast targets, or noise measurements, while maintaining perceived brightness by increasing pixel drive or illumination amplitude. In one embodiment, even in bright productivity scenes, CADC gradually reduces the duty cycle to a lower limit of approximately 20 percent to limit image persistence and improve the readability of midtones; this reduction can be smoothed over several frames to avoid visible pumping. In game scenes with darker color palettes, the duty cycle reduction can be deeper, resulting in backlight power savings of approximately 0.12 W to approximately 0.46 W in representative titles measured in experiments, while maintaining scene brightness through compensation. CADC can be implemented in parallel with compositor rendering, so its computation does not increase pipeline latency.

[0076] In one embodiment, a lightweight statistical buffer of approximately 387×207 pixels, or another reduced resolution suitable for the panel, is sampled to estimate scene characteristics, and the CADC parameters are updated from these estimates as the compositor prepares for the next frame. CPU load can be minimized, and GPU cost per frame can be minimal, as the operation is primarily histogram or average brightness estimation. In a variant, the CADC weights can be learned or adaptively tuned based on user preferences, device thermal state, or measured line time margins.

[0077] In a user-adjustable embodiment, CADC discloses an intensity parameter that allows end users to tune the degree of duty cycle reduction relative to the compensation gain. Users who prefer to retain maximum detail in shadow areas can reduce the intensity so that compensation doesn't over-enhance low brightness, while users sensitive to motion blur can increase the intensity to benefit from lower image persistence. Profiles can be applied by application category or scene category, and can be overridden by platform policies when an application declares preferred rendering characteristics.

[0078] Global white point offset calibration can be performed to correct thermally induced color shifts from module conditions (e.g., about 25°C) to typical head-mounted conditions (e.g., about 40°C). In one embodiment, engineering builds and product samples are measured across hundreds of units to determine the statistical distribution of white point drift as a function of temperature and operating conditions. A global offset is then selected to focus the desired distribution onto the target white point, thereby improving color accuracy in VR rendering and pass-through camera pipelines.

[0079] Global offsets can be applied per program or per device. In some embodiments, global offsets are used to minimize manufacturing complexity, while in advanced device or service modes, per-unit calibration can be used to further reduce color errors. Per-unit calibration can be performed during line-end testing by applying a device-specific offset derived from measured panel behavior and stored in non-volatile memory; runtime calibration can also be supported when the sensor detects sustained thermal conditions beyond the expected range. Latency reduction can be achieved through multiple coordinated knobs. In one embodiment, color difference correction (CAC) is performed on the display processing unit within the system-on-chip, such that the CAC output is fed directly to panel writing without round trips to a separate computation stage, thereby reducing the path from head orientation to photons by approximately 1 millisecond in a representative pipeline. The scheduler bases its refresh state estimate on post-CAC timing to align the window layout with actual write completion and ensure that inter-eye coordination remains accurate.

[0080] In another embodiment, the MIPI-DSI link rate is increased during high-speed motion scenarios or when the scheduler anticipates a tight window budget. By shortening the line time, the system expands the available margin between refresh completion and the next frame boundary, thereby achieving an earlier window start time. This earlier window start time reduces motion-to-photon latency while still meeting stability criteria. Link rate adjustments can be controlled through thermal space, power stability, or policy restrictions to avoid adverse effects on the device.

[0081] Display scaling is an optional pipeline adjustment that can affect refresh state timing and window budget. In one embodiment, dynamic resolution scaling reduces the number of pixels written per frame during heavy motion or when computationally loaded, which shortens the effective scan duration and increases the available time of the luminous window. Although scaling may be deferred or disabled in some programs, it remains a viable alternative to balance latency and artifact reduction without altering the underlying scheduling framework.

[0082] Brightness difference calibration can be performed between regions to equalize perceived brightness across the eye. In one embodiment, optical sensors located within the optical path measure the output from each illumination subsystem during calibration mode, and the controller fine-tunes the current or pulse amplitude to minimize the difference. In another embodiment, the difference is inferred from sensorless metrics such as driver telemetry and historical duty cycle levels, and calibration is applied via a PWM duty cycle or current proportional table. Calibration can be static at manufacturing time or dynamic during operation, with safeguards to maintain temporal coordination between eyes.

[0083] The disclosed scheduling, CADC, and calibration techniques encompass various display technologies. In LCOS embodiments, the illumination subsystem can be a projector light source driven by gating pulses synchronized with the shutter or digital micromirror position; the stabilization model reflects optical and mechanical dynamics, rather than the liquid crystal response. In microLED or silicon-based OLED embodiments, the illumination subsystem can be an on-board light-emitting driver with segmented gating, and the response model can depend on light emission decay and driver characteristics. Although the physical principles differ, the concepts of refresh state sensing for each region, operational condition-aware window layout, and inter-eye alignment remain applicable. These techniques are compatible with pass-through pipelines and can improve perceived sharpness and color fidelity in mixed reality scenarios. For example, CADC can reduce image persistence during camera-generated overlays, thereby enhancing edge sharpness and readability of UI elements, while global white point offset maintains natural color rendering in the real world. Light emission inter-eye alignment ensures that pass-through content appears temporarily stable across both eyes, even when scanning directions differ due to panel rotation.

[0084] In some embodiments, machine learning models are trained to predict stable intervals or optimal window positions as functions of temperature, voltage, refresh rate, content category, and historical device behavior. These models can replace or enhance analytical response curves and can be updated via over-the-air software updates as fleet data accumulates. Regardless of the implementation, the scheduler enforces safeguards (e.g., minimum stable intervals and maximum interocular offset) to maintain visual comfort. In other embodiments, a developer-facing API exposes configuration parameters including alignment tolerances, duty cycle lower limits, compensation gain, and pipeline delay offset. Applications can declare preferences for motion blur and brightness, and the scheduler reconciles these preferences within device policy constraints. User-oriented controls can be provided for CADC intensity or color warmth, with safe ranges that maintain artifact suppression and binocular comfort.

[0085] Measurement and feedback can be integrated to maintain performance throughout the device's lifespan. For example, as the panel ages and the LCD or LED response drifts, the system can gradually increase the modeling stabilization interval or adjust the compensation gain to keep artifacts below the perception threshold. Sensor readings for temperature, supply voltage, and timing drift can be fused into a stability metric that informs the window layout. Finally, the disclosed method can be combined with rendering optimization, such as concave rendering or gaze-based refresh prioritization. When eye tracking is available, the scheduler can bias the window layout towards regions of interest to reduce latency in areas of highest visual acuity while still satisfying inter-eye alignment globally. This variation is consistent with the core concept of determining the illumination window for each region from refresh status and operating conditions, and coordinating emission timing to align with perception and artifact suppression.

[0086] In summary, the disclosed scheduling architecture provides a concrete technical solution to a recognized technical problem in near-eye displays: simultaneously suppressing LCD ghosting and reducing motion-to-photon latency on a single panel without causing inter-eye timing discrepancies. By sensing operating conditions (e.g., temperature), tracking region-specific refresh states, and calculating illumination windows for each region, which are coordinated for inter-eye alignment and deferred until stability criteria are met, the system resolves conflicting constraints that existing designs only partially address or that are solved with dual-panel hardware. Conditional adaptation and per-eye emission control, achieved through an independently controllable illumination subsystem, directly improve temporal presentation fidelity, mitigate display-induced motion artifacts, and maintain user comfort during dynamic motion. Optional integrations (including content-adaptive duty cycle and global color shift) further improve efficiency and color accuracy without adding latency. Overall, these mechanisms convert raw panel and sensor signals into precise, synchronized illumination timing, resulting in measurable improvements in visual fidelity and system responsiveness through a well-defined technical approach.

[0087] Example Implementation Article 1. A computer-implemented method comprising: driving a near-eye display including a single image panel divided into a left-eye region and a right-eye region, and a corresponding independently controllable illumination subsystem; sensing operating conditions of the near-eye display; determining an illumination window for the region based at least in part on the refresh state and operating conditions of each of the left-eye and right-eye regions; and commanding the illumination subsystem to emit light during the determined illumination window while coordinating the emission timing of each region to satisfy inter-eye alignment and mitigate motion artifacts caused by the display.

[0088] Article 2. The method according to Article 1, wherein the operating conditions include at least temperature.

[0089] Article 3. The method according to Article 1, wherein the operating conditions include at least one of the following: ambient temperature; panel temperature; refresh rate; power supply voltage; lighting subsystem temperature; or panel age.

[0090] Article 4. The method according to Article 1, wherein the refresh status includes at least one of the following: scan output position; row or row index; region update complete indicator; or frame boundary indicator.

[0091] Article 5. The method according to Article 1 further comprises: for each of the left eye region and the right eye region, modeling the response characteristics of the image panel as a function of the operating conditions; and determining the illumination window based at least in part on the settling time obtained from the model.

[0092] Article 6. The method described in Article 1, wherein coordinating the emission timing of each region includes: enforcing interocular temporal parallax constraints.

[0093] Article 7. The method according to Article 6, wherein the interocular temporal parallax constraint includes the maximum permissible time offset between illumination windows.

[0094] Article 8. The method according to Article 1, wherein the lighting subsystem includes an independently controllable backlight.

[0095] Article 9. The method described in Article 1, wherein the lighting subsystem includes segmented lighting sources that are independently gated for the left and right eye regions.

[0096] Article 10. The method according to Article 1, wherein multiple lighting windows overlap in time.

[0097] Article 11. The method according to Article 1 further includes: adaptively modulating the illumination duty cycle within at least one illumination window.

[0098] Article 12. The method according to Article 11 further includes: compensating for perceived brightness by adjusting the effective brightness via at least one of pixel driving or illumination amplitude in response to duty cycle modulation.

[0099] Article 13. The method according to Article 1 further includes: sensing the motion state of the near-eye display or the user's head posture, and adjusting the emission timing of each region based at least in part on the motion state.

[0100] Article 14. The method according to Article 1, wherein sensing operating conditions includes: reading at least one temperature sensor coupled to the image panel or the illumination subsystem.

[0101] Article 15. The method according to Article 1 further includes: reducing motion-to-photon delay by advancing at least one illumination window while maintaining inter-eye alignment.

[0102] Article 16. The method described in Article 1 further includes: adjusting the data link rate or pipeline scheduling to align the refresh state with the coordinated luminescence timing.

[0103] Article 17. The method described in Article 1 further includes: performing color difference correction and coordinating the emission timing of each region to address processing delays.

[0104] Article 18. The method according to Article 1, wherein the single image panel is rotated relative to the optical axis such that the refresh state of the left eye region and the refresh state of the right eye region are performed in opposite directions.

[0105] Article 19. A system comprising: at least one physical processor; physical memory including computer-executable instructions, which, when executed by the physical processor, cause the physical processor to: drive a near-eye display including a single image panel divided into a left-eye region and a right-eye region, and a corresponding independently controllable illumination subsystem; sense operating conditions of the near-eye display; determine an illumination window for each region based at least in part on the refresh state and operating conditions of each region; and command the illumination subsystem to emit light during the determined illumination window while coordinating the emission timing of each region to achieve inter-eye alignment and mitigate motion artifacts caused by the display.

[0106] Article 20. A non-transitory computer-readable medium includes one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to: drive a near-eye display including a single image panel divided into left-eye and right-eye regions and a corresponding independently controllable illumination subsystem; sense operating conditions of the near-eye display; determine an illumination window for each region based at least in part on the refresh state and operating conditions of that region; and command the illumination subsystem to emit light during the determined illumination window while coordinating the emission timing of each region to satisfy inter-eye alignment and mitigate motion artifacts caused by the display.

[0107] The embodiments of this disclosure may include various types of Artificial-Reality (AR) systems, or combinations thereof. AR can be any overlaid functionality and / or sensorily detectable content presented by an AR system within a user's physical environment. In other words, AR is a form of reality that has been adjusted in some way before being presented to the user. AR may include and / or represent virtual reality (VR), augmented reality, mixed AR (MAR), or some combination and / or variation of these types of reality. Similarly, AR environments may include: VR environments (including non-immersive VR environments, semi-immersive VR environments, and fully immersive VR environments); augmented reality environments (including marked augmented reality environments, unmarked augmented reality environments, location-based augmented reality environments, and projection-based augmented reality environments); mixed reality environments; and / or any other type or form of mixed reality environment or alternative reality environment.

[0108] AR content can include entirely computer-generated content or computer-generated content combined with acquired (e.g., real-world) content. Such AR content can include video, audio, haptic feedback, or some combination thereof, any of which can be presented in a single channel or multiple channels (e.g., stereoscopic video that creates a three-dimensional (3D) effect for the viewer). Additionally, in some embodiments, AR can also be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality and / or otherwise for use in artificial reality (e.g., to perform activities in artificial reality).

[0109] AR systems can be implemented in a variety of different shapes and configurations. Some AR systems can be designed to operate without a near-eye display (NED). Other AR systems may include NEDs that also provide visibility into the real world (e.g., Figure 11 Augmented reality systems (1100) or NEDs that visually immerse users in artificial reality (e.g., Figure 12A and Figure 12B (Virtual Reality System 1200). While some AR devices may be standalone systems, others may communicate with and / or collaborate with external devices to provide an AR experience to the user. Examples of such external devices include handheld controllers, mobile devices, desktop computers, devices worn by the user, devices worn by one or more other users, and / or any other suitable external system.

[0110] Figures 5 to 8B An example artificial reality (AR) system according to some embodiments is shown. Figure 5 A first AR system 500 and a first example user interaction are shown, which uses a wrist wearable device 502, a head wearable device (e.g., AR glasses 504) and / or a handheld intermediary processing device (HIPD) 506. Figure 6 A second AR system 600 and a second example user interaction are shown, which use a wrist wearable device 602, AR glasses 604 and / or HIPD 606. Figure 7A and Figure 7B The interaction between AR system 700 and a third example user 708 is shown, which uses a wrist wearable device 702, a head wearable device (e.g., a VR headset 750) and / or a HIPD 706. Figure 8A and Figure 8BThe interaction between the fourth AR system 800 and the fourth example user 808 is shown, which uses a wrist wearable device 830, a VR headset 820 and / or a haptic device 860 (e.g., wearable haptic gloves).

[0111] The following is for reference. Figure 9 and Figure 10 To describe a wrist-worn wearable device 900 that can be used in wrist-worn wearable devices 502, 602, 702, 830 and one or more of their components; see below for reference. Figures 11 to 13 Similar descriptions may be used for AR glasses 504, 604 or VR headsets 750, 820, head-worn devices 1100 and 1200, and one or more of their components.

[0112] refer to Figure 5 The wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506 can be communicatively coupled via a network 525 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless local area network (LAN), etc.). Furthermore, the wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506 can also be communicatively coupled via a network 525 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN, etc.) to one or more servers 530, one or more computers 540 (e.g., laptops, computers, etc.), mobile devices 550 (e.g., smartphones, tablets, etc.), and / or other electronic devices.

[0113] exist Figure 5 The image shows a user 508 wearing a wrist-worn wearable device 502 and AR glasses 504, with a HIPD 506 placed on their table. The wrist-worn wearable device 502, AR glasses 504, and HIPD 506 facilitate user interaction with the AR environment. Specifically, as shown in the first AR system 500, the wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506 enable the presentation of one or more avatars 510, digital representations 512 of one or more contacts, and one or more virtual objects 514. As discussed below, the user 508 can interact with the one or more avatars 510, the digital representations 512 of one or more contacts, and the one or more virtual objects 514 through the wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506.

[0114] User 508 may use any of the wrist wearable device 502, AR glasses 504, and / or HIPD 506 to provide user input. For example, user 508 may perform actions by the wrist wearable device 502 (e.g., using one or more EMG sensors and / or IMU, as referred to below). Figure 9 and Figure 10(Description) and / or AR glasses 504 (e.g., using one or more image sensors or cameras, see below for reference) Figures 11 to 13 (Description) Detects one or more gestures to provide user input. Alternatively or additionally, user 508 may provide user input via: one or more touch surfaces of the wrist wearable device 502, AR glasses 504, or HIPD 506; and / or voice commands acquired by the microphones of the wrist wearable device 502, AR glasses 504, and / or HIPD 506. In some embodiments, the wrist wearable device 502, AR glasses 504, and / or HIPD 506 includes a digital assistant to assist user 508 in providing user input (e.g., completing a sequence of actions, suggesting different actions or commands, providing reminders, confirming commands). In some embodiments, user 508 may provide user input via one or more facial gestures and / or facial expressions. For example, the cameras of the wrist wearable device 502, AR glasses 504, and / or HIPD 506 may track user 508's eyes for navigating the user interface.

[0115] The wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506 can operate individually or collaboratively to allow user 508 to interact with the AR environment. In some embodiments, HIPD 506 is configured to operate as a central hub or control center for the wrist-worn wearable device 502, AR glasses 504, and / or another communicatively coupled device. For example, user 508 can provide input at any of the wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506 to interact with the AR environment, and HIPD 506 can identify one or more backend and frontend tasks to perform the requested interaction and distribute instructions to execute the one or more backend and frontend tasks at the wrist-worn wearable device 502, AR glasses 504, and / or HIPD 506. In some embodiments, backend tasks are user-insensible background processing tasks (e.g., rendering content, decompressing, compressing, etc.), and frontend tasks are user-insensible user-facing tasks (e.g., presenting information to the user, providing feedback to the user, etc.). As described below, HIPD 506 can perform backend tasks and provide operational data corresponding to the performed backend tasks to the wrist wearable device 502 and / or AR glasses 504, enabling the wrist wearable device 502 and / or AR glasses 504 to perform frontend tasks. In this way, HIPD 506 (which has more computing resources and a larger thermal headroom than the wrist wearable device 502 and / or AR glasses 504) performs computationally intensive tasks and reduces the computing resource utilization and / or power consumption of the wrist wearable device 502 and / or AR glasses 504.

[0116] In the example shown in the first AR system 500, HIPD 506 identifies one or more backend and frontend tasks associated with a user request initiating an AR video call with one or more other users (represented by avatar 510 and digital representations of contacts 512), and issues instructions to execute the one or more backend and frontend tasks. Specifically, HIPD 506 performs backend tasks for processing and / or rendering image data (and other data) associated with the AR video call, and provides AR glasses 504 with operational data associated with the performed backend tasks, causing AR glasses 504 to perform frontend tasks for presenting the AR video call (e.g., presenting avatar 510 and digital representations of contacts 512).

[0117] In some embodiments, HIPD 506 can function as a focal point or anchor point for presenting information. This allows user 508 to generally know where the information is presented. For example, as shown in the first AR system 500, avatar 510 and digital representations 512 of contacts are presented above HIPD 506. Specifically, HIPD 506 and AR glasses 504 work together to determine the location for presenting avatar 510 and digital representations 512 of contacts. In some embodiments, information can be presented at a predetermined distance from HIPD 506 (e.g., within 5 meters). For example, as shown in the first AR system 500, virtual object 514 is presented on a table at a distance from HIPD 506. Similar to the examples above, HIPD 506 and AR glasses 504 can work together to determine the location for presenting virtual object 514. Alternatively, in some embodiments, the presentation of information is not constrained by HIPD 506. More specifically, the avatar 510, the digital representation of the contact 512, and the virtual object 514 do not need to be presented within the predetermined distance of HIPD 506.

[0118] The user input provided at the wrist wearable device 502, AR glasses 504, and / or HIPD 506 is coordinated to enable the user to initiate, continue, and / or complete an operation using any device. For example, user 508 may provide user input to AR glasses 504 to cause AR glasses 504 to present a virtual object 514, and while AR glasses 504 presents the virtual object 514, user 508 may provide one or more gestures via wrist wearable device 502 to interact with and / or manipulate the virtual object 514.

[0119] Figure 6The image shows user 608 wearing wrist wearable device 602 and AR glasses 604, and holding HIPD 606. In the second AR system 600, wrist wearable device 602, AR glasses 604, and / or HIPD 606 are used to receive one or more messages and / or provide one or more messages to user 608's contacts. Specifically, wrist wearable device 602, AR glasses 604, and / or HIPD 606 detect and coordinate one or more user inputs to initiate a messaging application and prepare to reply to messages received through the messaging application.

[0120] In some embodiments, user 608 launches an application on wrist wearable device 602, AR glasses 604, and / or HIPD 606 via user input, causing the application to launch on at least one device. For example, in a second AR system 600, user 608 performs a gesture associated with a command to launch a messaging application (represented by messaging user interface 616); wrist wearable device 602 detects the gesture; and based on determining that user 608 is wearing AR glasses 604, wrist wearable device 602 causes AR glasses 604 to present the messaging user interface 616 of the messaging application. AR glasses 604 may present the messaging user interface 616 to user 608 via its display (e.g., as shown in user 608's field of view 618). In some embodiments, the application launches and runs on a device (e.g., wrist wearable device 602, AR glasses 604, and / or HIPD 606) that detects user input to launch the application, and that device provides operational data to another device to cause the messaging application to be presented. For example, the wrist-worn wearable device 602 can detect user input to launch a messaging application, launch and run the messaging application, and provide operational data to the AR glasses 604 and / or HIPD 606 to enable the presentation of the messaging application. Alternatively, the application can be launched and run on a different device than the one that detected the user input. For example, the wrist-worn wearable device 602 can detect gestures associated with launching the messaging application and enable the HIPD 606 to run the messaging application and coordinate its presentation.

[0121] Furthermore, user 608 can provide user input at wrist wearable device 602, AR glasses 604, and / or HIPD 606 to continue and / or complete an operation initiated on another device. For example, after launching a messaging application via wrist wearable device 602 and while the messaging user interface 616 is displayed on AR glasses 604, user 608 can provide input at HIPD 606 to prepare a reply (e.g., as shown by a swipe gesture performed on HIPD 606). Gestures performed by user 608 on HIPD 606 can be provided and / or displayed on another device. For example, a swipe gesture performed on HIPD 606 is displayed on the virtual keyboard of the messaging user interface 616 displayed by AR glasses 604.

[0122] In some embodiments, the wrist wearable device 602, AR glasses 604, HIPD 606, and / or other communication-coupled devices may present one or more notifications to the user 608. The notification may be an indication of a new message, incoming call, application update, or status update, etc. The user 608 may select the notification via the wrist wearable device 602, AR glasses 604, and / or HIPD 606, and may cause the application or action associated with the notification to be presented on at least one device. For example, the user 608 may receive a notification of receiving a message at the wrist wearable device 602, AR glasses 604, HIPD 606, and / or other communication-coupled devices, and may provide user input at the wrist wearable device 602, AR glasses 604, and / or HIPD 606 to view the notification. The device that detects the user input may cause the application associated with the notification to be launched and / or the application associated with the notification to be presented on the wrist wearable device 602, AR glasses 604, and / or HIPD 606.

[0123] While the examples above describe coordinated input for interaction with messaging applications, user input can be coordinated to interact with any number of applications, including but not limited to gaming applications, social media applications, camera applications, web-based applications, and financial applications. For example, AR glasses 604 can present game application data to user 608, and HIPD 606 can be used as a controller to provide input to the game. Similarly, user 608 can use wrist wearable device 602 to activate the camera of AR glasses 604, and user 608 can use wrist wearable device 602, AR glasses 604, and / or HIPD 606 to manipulate image acquisition (e.g., zoom in or zoom out, apply filters, etc.) and acquire image data.

[0124] Users can interact with the devices disclosed in this article in a variety of ways. For example, such as Figure 7A and Figure 7B As shown, user 708 can interact with AR system 700 by wearing VR headset 750, holding HIPD 706, and wearing wrist wearable device 702. In this example, AR system 700 allows users to interact with game 710 by waving their arms. One or more of VR headset 750, HIPD 706, and wrist wearable device 702 can detect the gesture and, in response, can display sword strikes in game 710. Similarly, in Figure 8A and Figure 8B In this example, user 808 can interact with AR system 800 by wearing VR headset 820 while simultaneously wearing haptic device 860 and wrist wearable device 830. In this example, AR system 800 allows the user to interact with game 810 by waving their arm. One or more of VR headset 820, haptic device 860, and wrist wearable device 830 can detect the gesture and, in response, display the spell being cast in game 810.

[0125] The following will now be discussed in more detail: the example AR systems already discussed more generally, devices for interacting with such AR systems, and other computing systems. For ease of reference, some interpretations of the following devices and components are defined herein: these devices and components may be included in some or all of the example devices discussed below. Certain types of components described below may be more suitable for a particular set of devices and less suitable for different sets of devices. However, subsequent references to components as explained herein should be considered as being covered by the interpretations provided.

[0126] In some of the embodiments discussed below, a number of example devices and systems, including electronic devices and systems, will be described. These example devices and systems are not intended to be limiting, and those skilled in the art will understand that alternative devices and systems to the example devices and systems described herein can be used to perform the various operations and construct the systems and devices described herein.

[0127] An electronic device can be a device that uses electrical energy to perform a specific function. An electronic device can be any physical object that contains electronic components, such as transistors, resistors, capacitors, diodes, and integrated circuits. Examples of electronic devices include smartphones, laptops, digital cameras, televisions, game consoles, and music players, as well as the example electronic devices discussed herein. As described herein, an intermediate electronic device is a device situated between two other electronic devices, and / or a subset of components of one or more electronic devices, and facilitates communication, data processing, and / or data transfer between the respective electronic devices and / or electronic components.

[0128] An integrated circuit (IC) can be an electronic device composed of multiple interconnected electronic components, such as transistors, resistors, and capacitors. These components can be etched onto small pieces of semiconductor material, such as silicon. ICs can include analog ICs, digital ICs, mixed-signal ICs, and / or any other suitable type or form of IC. Examples of ICs include application-specific integrated circuits (ASICs), processing units, central processing units (CPUs), coprocessors, and accelerators.

[0129] Analog integrated circuits (e.g., sensors, power management circuits, and operational amplifiers) can process continuous signals and perform analog functions such as amplification, active filtering, demodulation, and mixing. Examples of analog integrated circuits include linear integrated circuits and radio frequency circuits.

[0130] Digital integrated circuits, which may be referred to as logic integrated circuits, may include microprocessors, microcontrollers, memory chips, interfaces, power management circuits, programmable devices, and / or any other suitable type or form of integrated circuit. In some embodiments, an example of an integrated circuit includes a central processing unit (CPU).

[0131] A processing unit (e.g., a CPU) can be an electronic component responsible for executing instructions and controlling the operation of an electronic device (e.g., a computer). Various types of processors exist, which can be used interchangeably or specifically required by the embodiments described herein. For example, a processor can be: (i) a general-purpose processor designed to perform a wide range of tasks, such as running software applications, managing operating systems, and performing arithmetic and logical operations; (ii) a microcontroller designed for specific tasks, such as controlling electronic devices, sensors, and motors; (iii) an accelerator, such as a graphics processing unit (GPU), designed to accelerate the creation and rendering of images, videos, and animations (e.g., virtual reality animations, such as 3D modeling); (iv) a field-programmable gate array (FPGA) that can be programmed and reconfigured post-manufacturing and / or customized to perform specific tasks, such as signal processing, encryption, and machine learning; and (v) a digital signal processor (DSP) designed to perform mathematical operations on signals (e.g., audio, video, and radio waves). One or more processors of one or more electronic devices may be used in the various embodiments described herein.

[0132] Memory generally refers to electronic components in a computer or electronic device that store data and instructions for access and manipulation by a processor. Examples of memory may include: (i) random access memory (RAM) configured to temporarily store data and instructions; (ii) read-only memory (ROM) configured to permanently store data and instructions and / or semi-permanently store data and instructions (e.g., one or more portions of system firmware and / or bootloader); (iii) flash memory (e.g., USB drives, memory cards, and / or solid-state drives (SSDs)) configured to store data in an electronic device; and / or (iv) cache memory configured to temporarily store frequently accessed data and instructions. As described herein, memory may include structured data (e.g., Structured Query Language (SQL) databases, MongoDB databases, GraphQL data, JSON data, etc.). Other examples of data stored in memory may include: (i) data data, including user account data, user settings, and / or other user data stored by the user; (ii) sensor data detected by one or more sensors and / or otherwise acquired; (iii) media content data, including stored image data, audio data, and documents; (iv) application data, which may include data collected and / or otherwise acquired and stored during use of the application; and / or any other types of data described herein.

[0133] A controller is an electronic component that manages and coordinates the operation of other components within an electronic device (e.g., controlling inputs, processing data, and / or generating outputs). Examples of controllers may include: (i) microcontrollers, which include small, low-power controllers commonly used in embedded systems and Internet of Things (IoT) devices; (ii) programmable logic controllers (PLCs), which can be configured for use in industrial automation systems to control and monitor manufacturing processes; (iii) system-on-a-chip (SoC) controllers, which integrate multiple components (e.g., processors, memory, I / O interfaces, and other peripherals) into a single chip; and / or (iv) digital signal processors (DSPs).

[0134] The power system of an electronic device can be configured to convert input power into a form usable for operating the device. The power system may include various components, such as: (i) a power source, which may be an alternating current (AC) adapter power source or a direct current (DC) adapter power source; (ii) a charger input, which may be configured to use wired and / or wireless connections (which may be part of a peripheral interface, such as USB, microUSB, near-field magnetic coupling, magnetic induction and magnetic resonance charging, and / or radio frequency (RF) charging); (iii) a power management integrated circuit configured to distribute power to various components of the device and ensure that the device operates within safety limits (e.g., regulating voltage, controlling current, and / or managing heat dissipation); and / or (iv) a battery configured to store power to provide usable power to components of one or more electronic devices.

[0135] Peripheral interfaces can be electronic components (e.g., of electronic devices) that allow the electronic device to communicate with other devices or peripheral devices and can provide the ability to input and output data and signals. Examples of peripheral interfaces may include: (i) a Universal Serial Bus (USB) and / or MicroUSB interface configured to connect a device to an electronic device; (ii) a Bluetooth interface configured to allow devices to communicate with each other, including Bluetooth Low Energy (BLE); (iii) a Near Field Communication (NFC) interface configured as a short-range wireless interface for operations such as access control; (iv) a POGO pin, which may be a small, spring-loaded pin configured to provide a charging interface; (v) a wireless charging interface; (vi) a Global Position System (GPS) interface; (vii) a Wi-Fi interface used to provide connectivity between the device and a wireless network; and / or (viii) a sensor interface.

[0136] Sensors can be electronic components configured to detect physical and environmental changes and generate electrical signals (e.g., electronic components in electronic devices (e.g., wearable devices) and / or electronic components that otherwise communicate electronically with electronic devices). Examples of sensors may include: (i) imaging sensors for collecting imaging data (e.g., including one or more cameras disposed on a corresponding electronic device); (ii) biopotential signal sensors; (iii) inertial measurement units (e.g., IMUs) for detecting changes in, for example, angular velocity, force, magnetic field, and / or acceleration; (iv) heart rate sensors for measuring a user's heart rate; (v) SpO2 sensors for measuring a user's blood oxygen saturation (SpO2) and / or other biometric data; (vi) capacitive sensors (e.g., sensor-skin interfaces) for detecting potential changes at a part of a user's body; and / or (vii) light sensors (e.g., time-of-flight sensors, infrared light sensors, visible light sensors, etc.).

[0137] Biopotential signal sensing components can be devices used to measure electrical activity within the body (e.g., biopotential signal sensors). Some types of biopotential signal sensors include: (i) electroencephalography (EEG) sensors configured to measure electrical activity in the brain to diagnose neurological disorders; (ii) electrocardiography (ECG or EKG) sensors configured to measure electrical activity in the heart to diagnose heart problems; (iii) electromyography (EMG) sensors configured to measure electrical activity in muscles and diagnose neuromuscular disorders; and (iv) electrooculography (EOG) sensors configured to measure electrical activity in eye muscles to detect eye movements and diagnose eye disorders.

[0138] Applications (e.g., software) stored in the memory of an electronic device may include instructions stored in the memory. Examples of such applications include: (i) games; (ii) word processors; (iii) messaging applications; (iv) media streaming applications; (v) financial applications; (vi) calendars; (vii) clocks; and (viii) communication interface modules (e.g., IEEE 1102.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART or MiWi, custom or standard wired protocols (e.g., Ethernet or HomePlug), and / or any other suitable communication protocols) for enabling wired and / or wireless connections between different corresponding electronic devices.

[0139] A communication interface can be a mechanism that enables different systems or devices to exchange information and data with each other, including hardware, software, or a combination of both. For example, a communication interface can refer to a physical connector and / or port on a device that enables communication with other devices (e.g., USB, Ethernet, HDMI, Bluetooth). In some embodiments, a communication interface can refer to a software layer that enables different software programs to communicate with each other (e.g., application programming interface (API), protocols such as HTTP and TCP / IP, etc.).

[0140] A graphics module can be a component or software module designed to handle graphics operations and / or graphical processes, and the graphics module may include hardware modules and / or software modules.

[0141] Non-transitory computer-readable storage media can be physical devices or storage media that can be used to store electronic data in a non-transitory form (e.g., such that the data is permanently stored until it is intentionally deleted or modified).

[0142] Figure 9 and Figure 10 An example wrist-worn wearable device 900 and an example computing system 1000 are illustrated according to some embodiments. The wrist-worn wearable device 900 is described herein. Figure 5 The example of wrist wearable device 502 described herein is such that wrist wearable device 502 should be understood as having the characteristics of wrist wearable device 900, and vice versa. Figure 10 Several components of a wrist-worn wearable device 900 are shown, which can be used individually or in combination, including combinations that include other electronic devices and / or electronic components.

[0143] Figure 9 A wearable strap 910 and a watch body 920 (or capsule) are shown coupled, as described below, to form a wrist-worn wearable device 900. The wrist-worn wearable device 900 can perform various functions and / or operations associated with browsing the user interface and selectively opening applications, as well as those described above. Figures 5 to 8B The described functions and / or operations.

[0144] As will be described in more detail below, the operations performed by the wrist-worn wearable device 900 may include: (i) presenting content to the user (e.g., displaying visual content via display 905); (ii) detecting (e.g., sensing) user input (e.g., sensing touches on peripheral buttons 923 and / or touches at the touchscreen of display 905, gestures detected by sensors (e.g., biopotential sensors); (iii) sensing biometric data (e.g., neuromuscular signals, heart rate, temperature, sleep, etc.) via one or more sensors 913; sending and receiving messages (e.g., text, voice, video, etc.); image acquisition via one or more imaging devices or cameras 925; wireless communication (e.g., cellular, near-field, Wi-Fi, personal area network, etc.); location determination; financial transactions; providing haptic feedback; providing warnings; providing notifications; providing biometric authentication; providing health monitoring; providing sleep monitoring, etc.

[0145] The example functions described above can be performed independently in the watch body 920, independently in the wearable band 910, and / or via electronic communication between the watch body 920 and the wearable band 910. In some embodiments, when an AR environment is presented (e.g., via one of AR systems 500 to 800), the functions can be performed on the wrist wearable device 900. The wearable device described herein can also be used in other types of AR environments.

[0146] The wearable band 910 can be configured to be worn by a user such that the inner surface of the wearable structure 911 of the wearable band 910 contacts the user's skin. In this example, the sensor 913 contacts the user's skin when worn by the user. In some examples, one or more of the multiple sensors 913 can sense biometric data, such as the user's heart rate, saturated oxygen level, temperature, sweat level, neuromuscular signals, or combinations thereof. One or more of the multiple sensors 913 can also sense data about the user's environment, including the user's motion, height, location, orientation, gait, acceleration, position, or combinations thereof. In some embodiments, one or more of the multiple sensors 913 can be configured to track the position and / or motion of the wearable band 910. One or more of the multiple sensors 913 can include those defined above and / or referenced below. Figure 9 Any of the multiple sensors discussed.

[0147] One or more of the plurality of sensors 913 may be distributed on the inner and / or outer surface of the wearable band 910. In some embodiments, one or more of the plurality of sensors 913 are evenly spaced along the wearable band 910. Alternatively, in some embodiments, one or more of the plurality of sensors 913 are positioned at different points along the wearable band 910. Figure 9 As shown, one or more of the plurality of sensors 913 may be the same or different. For example, in some embodiments, one or more of the plurality of sensors 913 may be shaped as a pill (e.g., sensor 913a), an egg, a circle, a square, an ellipse (e.g., sensor 913c), and / or any other shape that remains in contact with the user's skin (e.g., so that neuromuscular signals and / or other biometric data can be accurately measured at the user's skin). In some embodiments, one or more sensors 913 are aligned to form sensor pairs (e.g., for sensing neuromuscular signals based on differential sensing within each respective sensor). For example, sensor 913b may be aligned with an adjacent sensor to form sensor pair 914a, and sensor 913d may be aligned with an adjacent sensor to form sensor pair 914b. In some embodiments, the wearable band 910 does not have sensor pairs. Alternatively, in some embodiments, the wearable band 910 has a predetermined number of sensor pairs (one pair, three pairs, four pairs, six pairs, sixteen pairs, etc.).

[0148] The wearable band 910 may include any suitable number of sensors 913. In some embodiments, the number and arrangement of the sensors 913 depend on the specific application using the wearable band 910. For example, the wearable band 910 may be configured as an armband, wristband, or chest band, which includes multiple sensors 913 with different numbers of sensors 913, various types of individual sensors with multiple sensors 913, and different arrangements for each use case (e.g., medical use cases compared to gaming or general everyday use cases).

[0149] According to some embodiments, the wearable band 910 also includes an electrically grounding electrode and a shielding electrode. Similar to the sensor 913, the electrically grounding electrode and the shielding electrode may be distributed on the inner surface of the wearable band 910 such that they contact a portion of the user's skin. For example, the electrically grounding electrode and the shielding electrode may be located on the inner surface of the coupling mechanism 916 or the inner surface of the wearable structure 911. The electrically grounding electrode and the shielding electrode may be formed of and / or use the same components as the sensor 913. In some embodiments, the wearable band 910 includes more than one electrically grounding electrode and more than one shielding electrode.

[0150] Sensor 913 may be formed as part of the wearable structure 911 of the wearable band 910. In some embodiments, sensor 913 is flush or substantially flush with the wearable structure 911, such that these sensors do not extend beyond the surface of the wearable structure 911. Although flush with the wearable structure 911, sensor 913 is still configured to contact the user's skin (e.g., through a skin-contact surface). Alternatively, in some embodiments, sensor 913 extends beyond the wearable structure 911 by a predetermined distance (e.g., 0.1 mm to 2 mm) to contact and press into the user's skin. In some embodiments, sensor 913 is coupled to an actuator (not shown) configured to adjust the extension height of sensor 913 (e.g., the distance from the surface of the wearable structure 911) such that sensor 913 contacts and presses into the user's skin. In some embodiments, the actuator adjusts the extension height between 0.01 mm and 1.2 mm. This allows the user to customize the position of sensor 913 to improve the overall comfort of the wearable band 910 when worn, while still allowing sensor 913 to contact the user's skin. In some embodiments, the sensor 913 is not distinguishable from the wearable structure 911 when worn by a user.

[0151] The wearable structure 911 may be formed of an elastic material, elastomer, etc., which is configured to be stretched and adapted for wear by a user. In some embodiments, the wearable structure 911 is a textile or woven fabric. As described above, the sensor 913 may be formed as part of the wearable structure 911. For example, the sensor 913 may be molded into the wearable structure 911, integrated into a woven fabric (e.g., the sensor 913 may be sewn into the fabric and mimic the flexibility of the fabric, and may and / or may consist of a series of woven fabric threads).

[0152] Wearable structure 911 may include flexible electronic connectors (hereinafter referred to as) that interconnect sensors 913, electronic circuits and / or other electronic components. Figure 10 (Description) These flexible electronic connectors are enclosed within the wearable band 910. In some embodiments, the flexible electronic connectors are configured to interconnect sensors 913, electronic circuitry, and / or other electronic components of the wearable band 910 with corresponding sensors and / or other electronic components of another electronic device (e.g., watch body 920). The flexible electronic connectors are configured to move with the wearable structure 911 such that adjustments made by the user to the wearable structure 911 (e.g., resizing, pulling, folding, etc.) do not exert pressure or tension on the electrical coupling of the components of the wearable band 910.

[0153] As described above, the wearable band 910 is configured to be worn by a user. Specifically, the wearable band 910 may be shaped or otherwise manipulated for wear by a user. For example, the wearable band 910 may be shaped to have a generally circular shape, such that the wearable band may be configured to be worn on the user's forearm or wrist. Alternatively, the wearable band 910 may be shaped to be worn on another part of the user's body (e.g., the user's upper arm (e.g., around the biceps), forearm, chest, leg, etc.). The wearable band 910 may include a retaining mechanism 912 (e.g., a buckle, hook-and-loop fastener, etc.) for securing the wearable band 910 to the user's wrist or other body part. While the wearable band 910 is worn by the user, the sensor 913 senses data from the user's skin (referred to as sensor data). In some examples, the sensor 913 of the wearable band 910 acquires (e.g., senses and records) neuromuscular signals.

[0154] Sensed data (e.g., sensed neuromuscular signals) can be used to detect and / or determine a user's intention to perform certain motor actions. In some examples, sensor 913 can sense and record neuromuscular signals from the user when the user performs muscle activation (e.g., movement, gestures, etc.). Detected and / or determined motor actions (e.g., phalanges (or fingers) movement, wrist movement, hand movement, and / or other muscle intentions) can be used to determine control commands or control information (instructions to execute certain commands after the data is sensed) for causing the computing device to execute one or more input commands. For example, sensed neuromuscular signals can be used to control certain user interfaces displayed on display 905 of a wrist-worn wearable device 900, and / or can be sent to a device responsible for rendering an artificial reality environment (e.g., a head-mounted display) to perform actions within the associated artificial reality environment, such as controlling the movement of a virtual device displayed to the user. Muscle activation performed by a user can include: static gestures, such as placing the user's palm down on a table; dynamic gestures, such as grasping a physical or virtual object; and covert gestures that are imperceptible to another person, such as slightly tensing a joint by coordinating the contraction of opposing muscles or using submuscular activation. Muscle activation performed by a user can also include symbolic gestures (e.g., gestures that are mapped to other gestures, interactions, or commands based on a gesture vocabulary that specifies a gesture as a command).

[0155] Sensor data sensed by sensor 913 can be used to provide users with enhanced interaction with physical objects (e.g., devices communicatively coupled to wearable belt 910) and / or virtual objects in artificial reality applications generated by artificial reality systems (e.g., user interface objects presented on display 905 or another computing device (e.g., smartphones)).

[0156] In some embodiments, the wearable band 910 includes one or more tactile devices 1046 (e.g., vibratory tactile actuators) configured to provide tactile feedback (e.g., skin sensation and / or kinesthetic sensation) to the user's skin. Sensors 913 and / or tactile devices 1046 (such as...) Figure 10 (As shown) can be configured to work with a variety of applications, including but not limited to health monitoring, social media, games and artificial reality (e.g., applications associated with artificial reality).

[0157] The wearable band 910 may also include a coupling mechanism 916 for detachably coupling a pod (e.g., a computing unit) or a watch body 920 (via a coupling surface of the watch body 920) to the wearable band 910. For example, the bracket or shape of the coupling mechanism 916 may correspond to the shape of the watch body 920 of the wrist wearable device 900. In particular, the coupling mechanism 916 may be configured to receive a coupling surface of the watch body 920 near its bottom side (e.g., the side opposite the front side where the display 905 of the watch body 920 is located), allowing a user to push the watch body 920 downward into the coupling mechanism 916 to attach the watch body 920 to the coupling mechanism 916. In some embodiments, the coupling mechanism 916 may be configured to receive a top side of the watch body 920 (e.g., the side near the front side where the display 905 of the watch body 920 is located), which is pushed upward into the bracket rather than downward into the coupling mechanism 916. In some embodiments, the coupling mechanism 916 is an integrated component of the wearable strap 910, such that the wearable strap 910 and the coupling mechanism 916 are a single unified structure. In some embodiments, the coupling mechanism 916 is a frame or housing of the type that allows the coupling surface of the watch body 920 to be held within or on the coupling mechanism 916 of the wearable strap 910 (e.g., a bracket, tracking strap, support base, buckle, etc.).

[0158] The coupling mechanism 916 allows the watch body 920 to be detachably coupled to the wearable strap 910 via friction engagement, magnetic coupling, a rotation-based connector, a shear pin coupler, a retaining spring, one or more magnets, a clamp, a pin, a hook-and-loop fastener, or a combination thereof. A user can perform any type of action to couple the watch body 920 to and detach it from the wearable strap 910. For example, a user can twist, slide, rotate, push, pull, or rotate the watch body 920 relative to the wearable strap 910 to attach and detach it from the wearable strap 910. Alternatively, as discussed below, in some embodiments, the watch body 920 can be detached from the wearable strap 910 by actuation of the release mechanism 929.

[0159] The wearable band 910 can be coupled to the watch body 920 to increase the functionality of the wearable band 910 (e.g., converting the wearable band 910 into a wrist-worn wearable device 900, adding additional computing units and / or batteries to increase the computing resources and / or battery life of the wearable band 910, adding additional sensors to improve sensed data, etc.). As described above, the wearable band 910 and the coupling mechanism 916 are configured to operate independently of the watch body 920 (e.g., perform functions independently of the watch body). For example, the coupling mechanism 916 may include one or more sensors 913 that contact the user's skin when the user wears the wearable band 910 and provide sensor data for determining control commands.

[0160] Users can detach the watch body 920 from the wearable strap 910 to reduce the burden of the wrist wearable device 900 on the user. In embodiments where the watch body 920 is detachable, the watch body 920 may be referred to as a detachable structure, such that in these embodiments, the wrist wearable device 900 includes a wearable portion (e.g., the wearable strap 910) and a detachable structure (watch body 920).

[0161] Turning to the watch body 920, in some examples, the watch body 920 may have a generally rectangular or circular shape. The watch body 920 is configured to be worn by a user on their wrist or on another body part. More specifically, the watch body 920 is sized for easy carrying by the user, easy attachment to a part of the user's clothing, and / or easy coupling to the wearable strap 910 (thus forming the wrist wearable device 900). As described above, the watch body 920 may have a shape corresponding to the coupling mechanism 916 of the wearable strap 910. In some embodiments, the watch body 920 includes a single release mechanism 929 or multiple release mechanisms (e.g., two release mechanisms 929 positioned on opposite sides of the watch body 920, such as spring-supported buttons) to detach the watch body 920 from the wearable strap 910. The release mechanism 929 may include, but is not limited to, buttons, knobs, plugs, handles, levers, fasteners, buckles, dials, latches, or combinations thereof.

[0162] The user can actuate the release mechanism 929 by pushing, rotating, lifting, pressing, moving, or performing other actions on it. Actuation of the release mechanism 929 can release (e.g., detach) the watch body 920 from the coupling mechanism 916 of the wearable band 910, thereby allowing the user to use the watch body 920 independently of the wearable band 910, and vice versa. For example, detaching the watch body 920 from the wearable band 910 allows the user to use the rear camera 925b to capture images. Although the release mechanism 929 is shown as being located at a corner of the watch body 920, it can be located anywhere on the watch body 920 that is convenient for user actuation. Additionally, in some embodiments, the wearable band 910 may also include a corresponding release mechanism for detaching the watch body 920 from the coupling mechanism 916. In some embodiments, the release mechanism 929 is optional, and the watch body 920 can be detached from the coupling mechanism 916 as described above (e.g., by twisting, rotating, etc.).

[0163] The watch body 920 may include one or more peripheral buttons 923 and 927 for performing various operations at the watch body 920. For example, peripheral buttons 923 and 927 may be used to turn on or wake (e.g., switch from a sleep state to an active state) the display 905, unlock the watch body 920, increase or decrease the volume, increase or decrease the brightness, interact with one or more applications, interact with one or more user interfaces, etc. Additionally or alternatively, in some embodiments, the display 905 acts as a touchscreen and allows the user to provide one or more inputs for interacting with the watch body 920.

[0164] In some embodiments, the watch body 920 includes one or more sensors 921. The sensors 921 of the watch body 920 may be the same as or different from the sensors 913 of the wearable strap 910. The sensors 921 of the watch body 920 may be distributed on the inner and / or outer surfaces of the watch body 920. In some embodiments, the sensors 921 are configured to contact the user's skin when the user wears the watch body 920. For example, the sensors 921 may be placed on the underside of the watch body 920, and the coupling mechanism 916 may be a bracket with an opening that allows the underside of the watch body 920 to directly contact the user's skin. Alternatively, in some embodiments, the watch body 920 does not include sensors configured to contact the user's skin (e.g., sensors including those inside and / or outside the watch body 920, configured to sense data from the watch body 920 and data from the surrounding environment). In some embodiments, the sensors 921 are configured to track the position and / or movement of the watch body 920.

[0165] The watch body 920 and the wearable band 910 can share data using wired communication methods (e.g., Universal Asynchronous Receiver / Transmitter (UART), USB transceiver, etc.) and / or wireless communication methods (e.g., Near Field Communication, Bluetooth, etc.). For example, the watch body 920 and the wearable band 910 can share data sensed by sensors 913 and 921, as well as application and device-specific information (e.g., active and / or available applications, output devices (e.g., display, speaker, etc.), input devices (e.g., touchscreen, microphone, imaging sensor, etc.)).

[0166] In some embodiments, the watch body 920 may include, but is not limited to, a front-facing camera 825a and / or a rear-facing camera 825b, and sensors 921 (e.g., biometric sensors, IMUs, heart rate sensors, oxygen saturation sensors, neuromuscular signal sensors, altimeter sensors, temperature sensors, bioimpedance sensors, pedometer sensors, optical sensors (e.g., imaging sensor 1063), touch sensors, sweat sensors, etc.). In some embodiments, the watch body 920 may include one or more tactile devices 1076 (e.g., vibratory tactile actuators) configured to provide tactile feedback to the user (e.g., skin sensation and / or kinesthetic sensation, etc.). Sensors 1021 and / or tactile devices 1076 may also be configured to operate in conjunction with multiple applications, including but not limited to health monitoring applications, social media applications, gaming applications, and artificial reality applications (e.g., applications associated with artificial reality).

[0167] As described above, the watch body 920 and the wearable strap 910, when coupled, can form a wrist wearable device 900. The watch body 920 and the wearable strap 910, when coupled, can function as a single device to perform the functions described herein (operation, detection, communication, etc.). In some embodiments, each device may be provided with specific instructions for performing one or more operations of the wrist wearable device 900. For example, if it is determined that the watch body 920 does not include a neuromuscular signal sensor, the wearable strap 910 may include alternative instructions for performing associated instructions (e.g., providing sensed neuromuscular signal data to the watch body 920 via another electronic device). The operations of the wrist wearable device 900 may be performed by the watch body 920 alone or by the watch body in conjunction with the wearable strap 910 (e.g., via a corresponding processor and / or hardware component), or vice versa. In some embodiments, the operation of the wrist wearable device 900, the watch body 920, and / or the wearable strap 910 may be performed in conjunction with one or more processors and / or hardware components.

[0168] See below for reference Figure 10 As described in the block diagram, the wearable band 910 and / or the watch body 920 may each include independent resources required to perform their functions independently. For example, the wearable band 910 and / or the watch body 920 may each include a power source (e.g., a battery), memory, data storage device, processor (e.g., a central processing unit (CPU)), communication, light source, and / or input / output devices.

[0169] Figure 10 Block diagrams are shown of a computing system 1030 corresponding to a wearable strap 910 and a computing system 1060 corresponding to a watch body 920, according to some embodiments. The computing system 1000 of the wrist wearable device 900 may include a combination of components of the computing system 1030 of the wearable strap and components of the computing system 1060 of the watch body.

[0170] The watch body 920 and / or wearable band 910 may include one or more components shown in the watch body's computing system 1060. In some embodiments, a single integrated circuit includes all or most of the components of the watch body's computing system 1060, which are included in a single integrated circuit. Alternatively, in some embodiments, the components of the watch body's computing system 1060 are included in multiple communication-coupled integrated circuits. In some embodiments, the watch body's computing system 1060 may be configured (e.g., via a wired or wireless connection) to couple with the wearable band's computing system 1030, which allows the two computing systems to share components, distribute tasks, and / or (individually or as a single device) perform other operations described herein.

[0171] The computing system 1060 of the meter may include one or more processors 1079, controllers 1077, peripheral interfaces 1061, power systems 1095, and memory (e.g., memory 1080).

[0172] The power system 1095 may include a charger input 1096, a power-management integrated circuit (PMIC) 1097, and a battery 1098. In some embodiments, the watch body 920 and the wearable strap 910 may have their own batteries (e.g., batteries 1098 and 1059) and may share power with each other. The watch body 920 and the wearable strap 910 may use various technologies to receive charge. In some embodiments, the watch body 920 and the wearable strap 910 may use wired charging components (e.g., a power cord) to receive charge. Alternatively or additionally, the watch body 920 and / or the wearable strap 910 may be configured for wireless charging. For example, a portable charging device may be designed to mate with a portion of the watch body 920 and / or a portion of the wearable strap 910 and wirelessly deliver available power to the battery 1098 of the watch body 920 and / or the battery 1059 of the wearable strap 910. The watch body 920 and the wearable band 910 may have independent power systems (e.g., power systems 1095 and 1056, respectively) to enable each to operate independently. The watch body 920 and the wearable band 910 may also share power through their respective PMICs (e.g., PMICs 1097 and 1058) (e.g., one can charge the other), which can share power through power conductors and ground conductors and / or through wireless charging antennas.

[0173] In some embodiments, the peripheral interface 1061 may include one or more sensors 1021. Sensor 1021 may include one or more coupling sensors 1062 for detecting when the watch body 920 is coupled to another electronic device (e.g., a wearable band 910). Each sensor 1021 may include one or more imaging sensors 1063 (one or more of a camera 1025 and / or a separate imaging sensor 1063 (e.g., a thermal imaging sensor)). In some embodiments, sensor 1021 may include one or more SpO2 sensors 1064. In some embodiments, each sensor 1021 may include one or more bioelectric potential signal sensors (e.g., an EMG sensor 1065, which may be disposed on the user-facing portion of the watch body 920 and / or the wearable band 910). In some embodiments, sensor 1021 may include one or more capacitive sensors 1066. In some embodiments, sensor 1021 may include one or more heart rate sensors 1067. In some embodiments, sensor 1021 may include one or more IMU sensors 1068. In some embodiments, one or more IMU sensors 1068 may be configured to detect movement of the user's hand, or movement of the watch body 920 in other positions where it is placed or held.

[0174] In some embodiments, one or more of the plurality of sensors 1021 may provide an example human-machine interface. For example, a set of neuromuscular sensors (e.g., EMG sensor 1065) may be arranged circumferentially around the wearable band 910, wherein the inner surface of the EMG sensor 1065 is configured to contact the user's skin. Any suitable number of neuromuscular sensors may be used (e.g., 2 to 20 sensors). The number and arrangement of the neuromuscular sensors may depend on the specific application for which the wearable device is used. For example, the wearable band 910 may be used to generate control information for controlling augmented reality systems, robots, vehicles, scrolling text, avatars, or any other suitable control task.

[0175] In some embodiments, flexible electronics integrated into a wireless device can be used to couple the neuromuscular sensors together, and optionally, hardware signal processing circuitry can be used to process the outputs of one or more of the sensing elements (e.g., to perform amplification, filtering, and / or rectification). In other embodiments, at least some of the signal processing on the outputs of the multiple sensing elements can be performed in software (e.g., processor 1079). Therefore, signal processing on the signals sampled by the sensors can be performed in hardware, software, or by any suitable combination of hardware and software, as the aspects of the techniques described herein are not limited in this respect.

[0176] Neuromuscular signals can be processed in various ways. For example, the output of the EMG sensor 1065 can be provided to an analog front end, which can be configured to perform analog processing (e.g., amplification, noise reduction, filtering, etc.) on the recorded signal. The processed analog signal can then be provided to an analog-to-digital converter (ADC) that can convert the analog signal into a digital signal, which can be processed by one or more computer processors. Furthermore, although this example is discussed in the context of an interface with an EMG sensor, the embodiments described herein can also be implemented in wearable interfaces with other types of sensors, including but not limited to mechanomyography (MMG) sensors, sonomyography (SMG) sensors, and electrical impedance tomography (EIT) sensors.

[0177] In some embodiments, each peripheral interface 1061 includes a near-field communication (NFC) component 1069, a global positioning system (GPS) component 1070, a long-term evolution (LTE) component 1071, and / or a Wi-Fi and / or Bluetooth communication component 1072. In some embodiments, the peripheral interface 1061 includes one or more buttons 1073 (e.g., Figure 9 The peripheral buttons 923 and 927 in the interface cause an operation to be performed on the body 920 when the user selects these buttons. In some embodiments, the peripheral interface 1061 includes one or more indicators, such as light-emitting diodes (LEDs), to provide the user with visual indicators (e.g., received messages, low battery, activated microphone and / or camera, etc.).

[0178] The watch body 920 may include at least one display 905 to show a user a visual representation of information or data, including user interface elements and / or three-dimensional virtual objects. The display may also include a touchscreen for inputting user input (e.g., touch gestures and swipe gestures). The watch body 920 may include at least one speaker 1074 and at least one microphone 1075 for providing audio signals to the user and receiving audio input from the user. The user can provide user input through the microphone 1075 and can also receive audio output from the speaker 1074 as part of a haptic event provided by the haptic controller 1078. The watch body 920 may include at least one camera 1025, including a front-facing camera 1025a and a rear-facing camera 1025b. The camera 1025 may include an ultra-wide-angle camera, a wide-angle camera, a fisheye camera, a spherical camera, a telephoto camera, a depth-sensing camera, or other types of cameras.

[0179] The computing system 1060 of the watch body may include one or more haptic controllers 1078 and associated components (e.g., haptic devices 1076) for providing haptic events (e.g., a vibrational sensation or audio output responding to an event at the watch body 920) at the watch body 920. The haptic controllers 1078 may communicate with one or more haptic devices 1076 (e.g., electroacoustic devices), including: speakers in one or more speakers 1074; and / or other audio components; and / or electromechanical devices that convert energy into linear motion (e.g., motors, electromagnetic coils, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device)). The haptic controllers 1078 can provide haptic events that a user of the watch body 920 can sense. In some embodiments, the one or more haptic controllers 1078 may receive input signals from an application in application 1082.

[0180] In some embodiments, the computing system 1030 of the wearable strap and / or the computing system 1060 of the watch body may include a memory 1080, which may be controlled by one or more storage controllers among a plurality of controllers 1077. In some embodiments, software components stored in the memory 1080 include one or more applications 1082 configured to perform operations at the watch body 920. In some embodiments, the one or more applications 1082 may include games, word processors, messaging applications, calling applications, web browsers, social media applications, media streaming applications, financial applications, calendars, clocks, etc. In some embodiments, software components stored in the memory 1080 include one or more communication interface modules 1083 as defined above. In some embodiments, software components stored in the memory 1080 include: one or more graphics modules 1084 for rendering, encoding, and / or decoding audio data and / or visual data; and one or more data management modules 1085 for collecting and organizing data 1087 stored in the memory 1080 and / or providing access to the data 1087 stored in the memory 1080. In some embodiments, one or more applications and / or one or more modules in application 1082 may work together to perform various tasks at table body 920.

[0181] In some embodiments, the software components stored in the memory 1080 may include one or more operating systems 1081 (e.g., a Linux-based operating system, an Android operating system, etc.). The memory 1080 may also include data 1087. The data 1087 may include data 1088A, sensor data 1089A, media content data 1090, and application data 1091.

[0182] It should be recognized that the computing system 1060 of the table body is an example of a computing system within the table body 920, and the table body 920 may have more or fewer components than those shown in the computing system 1060 of the table body, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the computing system 1060 of the table body are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits (ASICs).

[0183] Turning to the computing system 1030 of the wearable strap, one or more components that may be included in the wearable strap 910 are shown. The computing system 1030 of the wearable strap may include more or fewer components than those shown in the computing system 1060 of the watch body, may combine two or more components, and / or may have different configurations and / or arrangements of some or all of these components. In some embodiments, all or most of the components of the multiple components of the computing system 1030 of the wearable strap are included in a single integrated circuit. Alternatively, in some embodiments, the multiple components of the computing system 1030 of the wearable strap are included in multiple communication-coupled integrated circuits. As described above, in some embodiments, the computing system 1030 of the wearable strap is configured to be coupled to the computing system 1060 of the watch body (e.g., via a wired or wireless connection), which allows the two computing systems to share components, assign tasks, and / or (individually or as a single device) perform other operations described herein.

[0184] Similar to the computing system 1060 of the watch body, the computing system 1030 of the wearable band may include: one or more processors 1049; one or more controllers 1047 (including one or more haptic controllers 1048); a peripheral interface 1031, which may include one or more sensors 1013 and other peripheral devices; a power supply (e.g., a power system 1056); and a memory (e.g., a memory 1050), which includes an operating system (e.g., an operating system 1051), data (e.g., data 1054, which includes data 1088B, sensor data 1089B, etc.) and one or more modules (e.g., a communication interface module 1052, a data management module 1053, etc.).

[0185] One or more of the multiple sensors 1013 may be similar to sensor 1021 of the computing system 1060 of the watch body. For example, sensor 1013 may include one or more coupling sensors 1032, one or more SpO2 sensors 1034, one or more EMG sensors 1035, one or more capacitive sensors 1036, one or more heart rate sensors 1037, and one or more IMU sensors 1038.

[0186] Peripheral interface 1031 may also include other components similar to those included in peripheral interface 1061 of the computing system 1060 of the watch body, as described above with reference to peripheral interface 1061. These other components include NFC component 1039, GPS component 1040, LTE component 1041, Wi-Fi and / or Bluetooth communication component 1042, and / or one or more haptic devices 1046. In some embodiments, peripheral interface 1031 includes one or more buttons 1043, a display 1033, a speaker 1044, a microphone 1045, and a camera 1055. In some embodiments, peripheral interface 1031 includes one or more indicators, such as LEDs.

[0187] It should be recognized that the computing system 1030 of the wearable band is an example of a computing system within the wearable band 910, and the wearable band 910 may have more or fewer components than those shown in the computing system 1030 of the wearable band, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the computing system 1030 of the wearable band can be implemented in one or more combinations of hardware, software, and firmware (including one or more signal processing and / or application-specific integrated circuits).

[0188] refer to Figure 9 The wrist wearable device 900 is an example of a wearable strap 910 and a watch body 920 coupled together, and therefore the wrist wearable device 900 will be understood to include components shown and described for a computing system 1030 for the wearable strap and a computing system 1060 for the watch body. In some embodiments, the wrist wearable device 900 has a split architecture (e.g., a split mechanical architecture, a split electrical architecture, etc.) between the watch body 920 and the wearable strap 910. In other words, all components shown in the computing system 1030 for the wearable strap and the computing system 1060 for the watch body can be accommodated or otherwise disposed in the combined wrist wearable device 900, or in individual components within the watch body 920, the wearable strap 910, and / or portions thereof (e.g., the coupling mechanism 916 of the wearable strap 910).

[0189] The above technology can be used with any device for sensing neuromuscular signals, but it can also be used with other types of wearable devices for sensing neuromuscular signals, such as body wearables or head wearables that may have neuromuscular sensors closer to the brain or spine.

[0190] In some embodiments, the wrist wearable device 900 may be used in conjunction with head wearable devices (e.g., AR system 1100 and VR system 1200) and / or HIPD as described below; and the wrist wearable device 900 may also be configured to allow a user to control any aspect of the artificial reality (e.g., by controlling user interface objects in the artificial reality using EMG-based gestures, and / or by allowing a user to interact with a touchscreen on the wrist wearable device to also control aspects of the artificial reality). Having thus described example wrist wearable devices, attention now turns to example head wearable devices, such as AR system 1100 and VR system 1200.

[0191] Figures 11 to 13 An example artificial reality system is shown, which can be used as or in conjunction with a wrist-worn wearable device 900. In some embodiments, the AR system 1100 includes, for example... Figure 11 The illustrated glasses device 1102. In some embodiments, the VR system 1210 includes a head-mounted display (HMD) 1212, such as... Figure 12A and Figure 12B As shown. In some embodiments, the AR system 1100 and VR system 1200 may include one or more similar components (e.g., components for presenting an interactive artificial reality environment, such as processors, memory, and / or presentation devices, including one or more displays and / or one or more waveguides), regarding Figure 13 Some of these components are described in more detail. As described herein, a head-mounted wearable device may include components of glasses device 1102 and / or components of head-mounted display 1212. Some embodiments of the head-mounted wearable device do not include any display (these displays include any displays described with respect to AR system 1100 and / or VR system 1200). Although the various example artificial reality systems are described herein as AR system 1100 and VR system 1200 respectively, any one or both of the various example AR systems described herein may be configured to present a fully immersive virtual reality scene within substantially the entire user's field of view, or to present a smaller augmented reality scene within a smaller portion of the user's field of view than the entire field of view.

[0192] Figure 11An example visual depiction of an AR system 1100 is shown, which includes a glasses device 1102 (which may also be described herein as augmented reality glasses and / or smart glasses). The AR system 1100 may include… Figure 11 Additional electronic components (e.g., wearable accessory devices and / or intermediate processing devices, not shown) that communicate electronically with or are otherwise configured to be used in conjunction with the eyewear device 1102. In some embodiments, the wearable accessory device and / or intermediate processing device may be configured to communicate with the coupling sensor 1324 (… Figure 13 The electronic communication coupling mechanism is coupled to the eyeglass device 1102, wherein the coupling sensor 1324 can detect when the electronic device becomes physically or electrically coupled to the eyeglass device 1102. In some embodiments, the eyeglass device 1102 may be configured to be coupled to the housing 1390. Figure 13 The housing 1390 may include one or more additional coupling mechanisms configured to couple with additional accessory devices. Figure 11 The components shown can be implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing components and / or application-specific integrated circuits (ASICs).

[0193] The eyewear device 1102 includes mechanical eyewear components, including a frame 1104 configured to hold one or more lenses (e.g., one or two lenses 1106-1 and 1106-2). Those skilled in the art will recognize that the eyewear device 1102 may include additional mechanical components, such as hinges configured to allow multiple portions of the frame 1104 of the eyewear device 1102 to fold and unfold, a bridge configured to span the gap between lenses 1106-1 and 1106-2 and rest on the user's nose, a nose pad configured to rest on the bridge of the nose and provide support for the eyewear device 1102, earpieces configured to rest on the user's ears and provide additional support for the eyewear device 1102, and temples configured to extend from the hinges of the eyewear device 1102 to the earpieces, etc. Those skilled in the art will further recognize that some examples of the AR system 1100 may not include the mechanical components described herein. For example, a smart contact lens configured to present an artificial reality to a user may not include any components of the eyewear device 1102.

[0194] Eyeglasses device 1102 includes electronic components, many of which will be referenced below. Figure 13 A more detailed description is provided. Some example electronic components are shown in... Figure 11As shown, the device includes acoustic sensors 1125-1, 1125-2, 1125-3, 1125-4, 1125-5, and 1125-6, which can be distributed along most of the frame 1104 of the eyewear device 1102. The eyewear device 1102 also includes a left camera 1139A and a right camera 1139B located on different sides of the frame 1104. The eyewear device 1102 also includes a processor 1148 (or any other suitable type or form of integrated circuit) embedded in a portion of the frame 1104.

[0195] Figure 12A and Figure 12B A VR system 1200, according to some embodiments, includes a head-mounted display (HMD) 1212 (e.g., also referred to herein as an artificial reality head-mounted device, a head-worn device, or a VR head-mounted device). As described above, some artificial reality systems (e.g., AR system 1100) can substantially replace one or more of a user's visual and / or other sensory perceptions of the real world with a virtual experience (e.g., AR systems 700 and 800), rather than mixing artificial reality with actual reality.

[0196] The HMD 1212 includes a front body 1214 and a frame 1216 (e.g., a strip or strap) shaped to fit the user's head. In some embodiments, the front body 1214 and / or frame 1216 include one or more electronic components (e.g., a display, IMU, tracking transmitter, or detector) for facilitating presentation to and / or interaction with AR and / or VR systems. In some embodiments, such as Figure 12B As shown, the HMD 1212 includes an output audio transducer (e.g., audio transducer 1218). In some embodiments, such as Figure 12B As shown, one or more components (e.g., one or more output audio transducers 1218 and frame 1216) (e.g., part or all of frame 1216, and / or audio transducer 1218) can be configured to be attached (e.g., detachably attached) to and removed from HMD 1212. In some embodiments, coupling a detachable component to HMD 1212 enables the detachable component to enter into electronic communication with HMD 1212.

[0197] Figure 12A and Figure 12BThe VR system 1200 is also shown to include one or more cameras, such as a left camera 1239A and a right camera 1239B, which may resemble the left camera 1139A and right camera 1139B on the frame 1104 of the glasses device 1102. In some embodiments, the VR system 1200 includes one or more additional cameras (e.g., cameras 1239C and 1239D) that may be configured to enhance the image data acquired by cameras 1239A and 1239B by providing more information. For example, camera 1239C may be used to provide color information not recognized by cameras 1239A and 1239B. In some embodiments, one or more of cameras 1239A to 1239D may include an optional IR cutoff filter configured to remove IR light from the light received from the respective camera sensor.

[0198] Figure 13 A computing system 1320 and an optional housing 1390 are shown, each of which illustrates components that may be included in the AR system 1100 and / or the VR system 1200. In some embodiments, depending on the actual constraints of the respective AR system described, more or fewer components may be included in the optional housing 1390.

[0199] In some embodiments, the computing system 1320 may include one or more peripheral interfaces 1322A and / or an optional housing 1390 may include one or more peripheral interfaces 1322B. Each of the computing system 1320 and the optional housing 1390 may also include one or more power systems 1342A and 1342B that can communicate electronically with each other, one or more controllers 1346 (including one or more haptic controllers 1347), one or more processors 1348A and 1348B (as defined above, including any examples provided), and memories 1350A and 1350B. For example, one or more processors 1348A and / or 1348B may be configured to execute instructions stored in memories 1350A and / or 1350B that may cause one of the one or more controllers 1346A to cause multiple operations to be performed at one or more peripheral devices of peripheral interfaces 1322A and / or 1322B. In some embodiments, each described operation may be powered by power supplied by power systems 1342A and / or 1342B.

[0200] In some embodiments, peripheral interface 1322A may include one or more devices configured as part of computing system 1320, some of which have been defined and / or referenced above. Figure 9 and Figure 10 The wrist-worn wearable device shown is described. For example, peripheral interface 1322A may include one or more sensors 1323A. Some example sensors 1323A include: one or more coupling sensors 1324, one or more acoustic sensors 1325, one or more imaging sensors 1326, one or more EMG sensors 1327, one or more capacitive sensors 1328, one or more IMU sensors 1329; and / or any other type of sensor explained above or described with respect to any other embodiments discussed herein.

[0201] In some embodiments, peripheral interfaces 1322A and 1322B may include one or more additional peripheral devices, including: one or more NFC devices 1330, one or more GPS devices 1331, one or more LTE devices 1332, one or more Wi-Fi and / or Bluetooth devices 1333, one or more buttons 1334 (e.g., including slide-able or otherwise adjustable buttons), one or more displays 1335A and 1335B, one or more speakers 1336A and 1336B, one or more microphones 1337, one or more cameras 1338A and 1338B (e.g., including left camera 1339A and / or right camera 1339B), one or more haptic devices 1340; and / or any other type of peripheral device defined above or described with respect to any other embodiments discussed herein.

[0202] AR systems can include various types of visual feedback mechanisms (e.g., presentation devices). For example, the display devices in AR system 1100 and / or VR system 1200 can include one or more liquid-crystal displays (LCDs), one or more light-emitting diode (LED) displays, one or more organic LED (OLED) displays, and / or any other suitable type of display. Artificial reality systems can include a single display (e.g., configured to be seen by both eyes), and / or can provide a separate display for each eye, which can allow for additional flexibility in zoom adjustment and / or correction of refractive errors associated with the user's vision. Some embodiments of AR systems also include an optical subsystem with one or more lenses (e.g., conventional concave or convex lenses, Fresnel lenses, or adjustable liquid lenses) through which the user views the display.

[0203] For example, corresponding displays 1335A and 1335B may be coupled to each of lenses 1106-1 and 1106-2 of the AR system 1100. Displays 1335A and 1335B coupled to each of lenses 1106-1 and 1106-2 may work together or independently to present an image or series of images to a user. In some embodiments, the AR system 1100 includes a single display 1335A or 1335B (e.g., a near-eye display) or more than two displays 1335A and 1335B. In some embodiments, one or more displays 1335A and 1335B from a first group may be used to present an augmented reality environment, and one or more display devices 1335A and 1335B from a second group may be used to present a virtual reality environment. In some embodiments, one or more waveguides (e.g., as a means of delivering light from one or more displays 1335A and 1335B to the user's eyes) are used in conjunction with presenting artificial reality content to a user of the AR system 1100. In some embodiments, one or more waveguides are wholly or partially integrated into the eyewear device 1102. As a supplement to or alternative to the display screen, some artificial reality systems include one or more projection systems. For example, the display device in AR system 1100 and / or VR system 1200 may include a microLED projector (e.g., using waveguides) that projects light onto the display device (e.g., a clear combiner lens that allows ambient light to pass through). The display device can refract the projected light into the user's pupil, allowing the user to simultaneously view both artificial reality content and the real world. Artificial reality systems can also be configured with any other suitable type or form of image projection system. In some embodiments, one or more waveguides are provided to supplement or replace one or more displays 1335A and 1335B.

[0204] The computing system 1320 of the AR system 1100 or VR system 1200 and / or the optional housing 1390 may include some or all of the components of the power systems 1342A and 1342B. The power systems 1342A and 1342B may include one or more charger inputs 1343, one or more PMICs 1344, and / or one or more batteries 1345A and 1344B.

[0205] Memory 1350A and 1350B may include instructions and data, some or all of which may be stored within memory 1350A and 1350B as a non-transitory computer-readable storage medium. For example, memory 1350A and 1350B may include one or more operating systems 1351, one or more applications 1352, one or more communication interface applications 1353A and 1353B, one or more graphics applications 1354A and 1354B, one or more AR processing applications 1355A and 1355B, and / or any other type of data as defined above or described with respect to any other embodiments discussed herein.

[0206] Memory 1350A and 1350B also include data 1360A and 1360B, which can be used in conjunction with one or more of the applications discussed above. Data 1360A and 1360B may include: data 1361; sensor data 1362A and 1362B; media content data 1363A; AR application data 1364A and 1364B; and / or any other type of data as defined above or described with reference to any other embodiments discussed herein.

[0207] In some embodiments, the controller 1346 of the glasses device 1102 can process information generated by sensors 1323A and / or 1323B on the glasses device 1102 and / or another electronic device within the AR system 1100. For example, the controller 1346 can process information from acoustic sensors 1125-1 and 1125-2. For each detected sound, the controller 1346 can perform direction of arrival (DOA) estimation to estimate the direction in which the detected sound arrives at the glasses device 1102 of the AR system 1100. When one or more acoustic sensors 1325 (e.g., acoustic sensors 1125-1, 1125-2) detect sound, the controller 1346 can use this information to populate an audio dataset (e.g., in...). Figure 13 (represented as sensor data 1362A and 1362B).

[0208] In some embodiments, physical electronic connectors can transmit information between the eyewear device 1102 and another electronic device and / or between one or more processors 1148, 1348A, 1348B and controller 1346 in the AR system 1100 or VR system 1200. This information can be in the following forms: optical data; electrical data; wireless data; or any other transmissible data form. Moving the processing of information generated by the eyewear device 1102 to an intermediate processing device can reduce the weight and heat of the eyewear device, making it more comfortable and safer for the user. In some embodiments, optional wearable accessory devices (e.g., electronic neckbands) are coupled to the eyewear device 1102 via one or more connectors. Each connector can be a wired or wireless connector and can include electronic and / or non-electronic components (e.g., structural components). In some embodiments, the eyewear device 1102 and the wearable accessory device can operate independently without any wired or wireless connection between them.

[0209] In some cases, external devices such as intermediate processing devices (e.g., HIPD 506, 606, 706) are paired with glasses devices 1102 (e.g., as part of AR system 1100) to enable glasses devices 1102 to achieve similar shape features to a pair of glasses while still providing sufficient battery and computing power for expanded capabilities. Some or all of the battery power, computing resources, and / or additional features of AR system 1100 can be provided by the paired device or shared between the paired device and glasses devices 1102, thus reducing the overall weight, heat profile, and shape features of glasses devices 1102 while allowing glasses devices 1102 to maintain their desired functionality. For example, wearable accessory devices can allow components that would otherwise be included on glasses devices 1102 to be included in the wearable accessory device and / or intermediate processing device, thereby transferring weight load from the user's head and neck to one or more other parts of the user's body. In some embodiments, the intermediate processing device has a large surface area to diffuse and disperse heat to the surrounding environment through this large surface area. Therefore, the intermediate processing device allows for greater battery and computing power compared to what might be available in other ways when using the glasses device 1102 alone. Because the weight carried by the wearable accessory device may have a smaller impact on the user than the weight carried by the glasses device 1102, users can tolerate wearing a lighter glasses device for longer periods while carrying or wearing paired devices, allowing the artificial reality environment to be more fully integrated into the user's daily activities, rather than tolerating wearing a heavier glasses device alone.

[0210] AR systems can include various types of computer vision components and subsystems. For example, AR system 1100 and / or VR system 1200 can include one or more optical sensors, such as two-dimensional (2D) or three-dimensional (3D) cameras, time-of-flight depth sensors, structured light emitters and detectors, single-beam or scanning laser rangefinders, 3D LiDAR sensors, and / or any other suitable type or form of optical sensor. AR systems can process data from one or more of these sensors to identify the user's location and / or multiple aspects of the user's real-world physical environment, including the location of real-world objects within the real-world physical environment. In some embodiments, the methods described herein are used to map the real world, provide the user with context about the real-world environment, and / or generate digital twins (e.g., interactive virtual objects), and various other functions. For example, Figure 12A and Figure 12B A VR system 1200 with cameras 1239A to 1239D is shown. The cameras 1239A to 1239D can be used to provide depth information, which is used to create a voxel field and a two-dimensional mesh to provide the user with object information to avoid collisions.

[0211] In some embodiments, the AR system 1100 and / or VR system 1200 may include haptic feedback systems that can be integrated into headwear, gloves, bodysuits, handheld controllers, environmental devices (e.g., chairs or footrests), and / or any other type of device or system (e.g., wearable devices discussed herein). Haptic feedback systems can provide various types of skin feedback, including vibration, force, tension, shear, texture, and / or temperature. Haptic feedback systems can also provide various types of kinematic feedback, such as motion and compliance. Haptic feedback can be implemented using motors, piezoelectric actuators, fluid systems, and / or various other types of feedback mechanisms. Haptic feedback systems can be implemented independently of other artificial reality devices, within other artificial reality devices, and / or in conjunction with other artificial reality devices.

[0212] In some embodiments of AR systems such as AR system 1100 and / or VR system 1200, ambient light (e.g., a live feed of the user's normally perceived surroundings) can pass through the display elements of the respective head-mounted wearable device presenting multiple aspects of the AR system. In some embodiments, ambient light can pass through a portion of the AR environment presented within the user's field of view that is less than the entire AR environment (e.g., a portion of the AR environment that is located in the same place as a physical object in the user's real-world environment, the physical object being within a specified boundary (e.g., a monitoring boundary) configured for use by the user when interacting with the AR environment). For example, visual user interface elements (e.g., notification user interface elements) can be presented on the head-mounted wearable device, and a certain amount of ambient light (e.g., 15% to 50% of the ambient light) can pass through the user interface element, allowing the user to distinguish the portion of the physical environment on which the user interface element is displayed.

[0213] In some examples, the augmented reality system described herein may also include a microphone array with multiple acoustic transducers. An acoustic transducer may represent a transducer that detects changes in air pressure caused by sound waves. Each acoustic transducer may be configured to detect sound and convert the detected sound into an electronic format (e.g., analog or digital). The microphone array may include, for example, ten acoustic transducers, which may be designed to be placed in the user's corresponding ear, or positioned at various locations on the HMD frame or strap.

[0214] In some embodiments, one or more of a plurality of acoustic transducers may be used as output transducers (e.g., speakers). For example, the artificial reality system described herein may include acoustic transducers that are earplugs or any other suitable type of headphones or speakers.

[0215] The configuration of the acoustic transducers in the microphone array can vary and can include any suitable number of transducers. In some embodiments, using a larger number of acoustic transducers can increase the amount of audio information collected and / or the sensitivity and accuracy of that information. Conversely, using a smaller number of acoustic transducers can reduce the computational power required by the associated controller to process the collected audio information. Furthermore, the position of each acoustic transducer in the microphone array can vary. For example, the position of the acoustic transducer can include a position defined on the user, coordinates defined on the frame of the HMD, an orientation associated with each acoustic transducer, or some combination thereof.

[0216] Acoustic transducers can be positioned at different locations on the user's ear, such as behind the outer ear, behind the tragus, and / or within the auricle or ear canal. Alternatively, additional acoustic transducers may be present on or around the ear in addition to those within the ear canal. Positioning the acoustic transducers close to the user's ear canal allows the microphone array to collect information about how sound reaches the ear canal. By positioning at least two of a plurality of acoustic transducers on either side of the user's head (e.g., as binaural microphones), the artificial reality device can simulate binaural hearing and capture a 3D stereo sound field around the user's head. In some embodiments, the acoustic transducers can be connected to the artificial reality system via a wired connection, while in other embodiments, they can be connected to the artificial reality system via a wireless connection (e.g., Bluetooth).

[0217] Acoustic transducers can be positioned on the HMD frame in a variety of different ways, including along the length of the temples, across the beam, above or below the display device, or some combination thereof. The acoustic transducers can also be oriented such that the microphone array can detect sound over a wide directional range around the user wearing the augmented reality system. In some embodiments, an optimization process can be performed during the manufacture of the augmented reality system to determine the relative positioning of the individual acoustic transducers within the microphone array.

[0218] The artificial reality system described herein may also include one or more input audio transducers and / or one or more output audio transducers. Output audio transducers may include voice coil loudspeakers, ribbon loudspeakers, electrostatic loudspeakers, piezoelectric loudspeakers, bone conduction transducers, cartilage conduction transducers, tragus vibration transducers, and / or any other suitable type or form of audio transducer. Similarly, input audio transducers may include condenser microphones, dynamic microphones, ribbon microphones, and / or any other type or form of input transducer. In some embodiments, a single transducer may be used for both audio input and audio output.

[0219] As described above, the computing devices and systems described and / or illustrated herein broadly refer to any type or form of computing device or system capable of executing computer-readable instructions (e.g., those contained within the modules described herein). In the most basic configuration of one or more computing devices, each of the one or more computing devices may include at least one storage device and at least one physical processor.

[0220] In some examples, the term "storage device" broadly refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a storage device may store, load, and / or maintain one or more of the modules described herein. Examples of storage devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache memory, variations or combinations of one or more of the above, or any other suitable storage memory.

[0221] In some examples, the term "physical processor" broadly refers to any type or form of hardware implementation of a processing unit capable of interpreting and / or executing computer-readable instructions. In one example, a physical processor can access and / or modify one or more modules stored in the aforementioned storage device. Examples of physical processors include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field-programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), portions of one or more of the above, variations or combinations of one or more of the above, or any other suitable physical processor.

[0222] Although the various modules described and / or illustrated herein are shown as individual elements, these modules may represent a portion of a single module or application. Furthermore, in some embodiments, one or more of these modules may represent one or more software applications or programs that, when executed by a computing device, enable the computing device to perform one or more tasks. For example, one or more of these modules described and / or illustrated herein may represent modules stored and configured to run on one or more of the computing devices or systems described and / or illustrated herein. One or more of these modules may also represent all or part of one or more dedicated computers configured to perform one or more tasks.

[0223] Furthermore, one or more of the modules described herein can transform data, physical devices, and / or representations of physical devices from one form to another. Additionally or alternatively, one or more of the modules stated herein can transform the processor, volatile memory, non-volatile memory, and / or any other part of the physical computing device from one form to another by executing on the computing device, storing data on the computing device, and / or otherwise interacting with the computing device.

[0224] In some embodiments, the term "computer-readable medium" broadly refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transport media (e.g., carrier waves), non-transitory media such as magnetic storage media (e.g., hard disk drives, magnetic tape drives, and floppy disks), optical storage media (e.g., optical discs (Compact Disk, CD, Digital Video Disk, DVD, and Blu-ray disc)), electronic storage media (e.g., solid-state drives and flash memory media), and other distributed systems.

[0225] The process parameters and sequence of steps described and / or illustrated herein are given by way of example only and may be changed as needed. For example, although the steps shown and / or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the shown or discussed order. The various exemplary methods described and / or illustrated herein may omit one or more of the steps described or illustrated herein, or may include additional steps in addition to those disclosed.

[0226] The preceding description has been provided to enable those skilled in the art to best utilize aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limited to any precise form disclosed. Many modifications and variations are possible without departing from the spirit and scope of this disclosure. The embodiments disclosed herein should be considered illustrative rather than restrictive in all respects. Reference should be made to the appended claims and their equivalents in determining the scope of this disclosure.

[0227] Unless otherwise stated, the terms “connected to” and “coupled to” (and their derivatives) as used in this specification and claims shall be interpreted as allowing direct and indirect (i.e., through other elements or components) connections. Additionally, the terms “a” or “an” as used in this specification and claims shall be interpreted as meaning “at least one of…”. Finally, for ease of use, the terms “comprising” and “having” (and their derivatives) as used in this specification and claims may be used interchangeably with the word “including” and have the same meaning as the word “comprising”.

Claims

1. A computer-implemented method, comprising: A near-eye display is driven, the near-eye display including a single image panel divided into a left-eye region and a right-eye region, and a corresponding independently controllable illumination subsystem; Sensing the operating conditions of the near-eye display; The illumination window for the region is determined at least in part based on the refresh state of each of the left and right eye regions and the operating conditions; as well as The illumination subsystem is instructed to emit light during the defined illumination window, while coordinating the timing of light emission in each region to achieve inter-eye alignment and reduce motion artifacts caused by the display.

2. The method according to claim 1, wherein, The operating conditions include at least temperature.

3. The method according to claim 1, wherein, The operating conditions include at least one of the following: ambient temperature; panel temperature; refresh rate; power supply voltage; lighting subsystem temperature; or panel age.

4. The method according to claim 1, wherein, The refresh status includes at least one of the following: scan output position; row or row index; region update complete indicator; or frame boundary indicator.

5. The method according to claim 1 further includes: For each of the left eye region and the right eye region, the response characteristics of the image panel are modeled as a function of the operating conditions; as well as The lighting window is determined at least in part based on the settling time obtained from the model.

6. The method according to claim 1, wherein, Coordinating the emission timing of each region includes: enforcing interocular temporal parallax constraints.

7. The method according to claim 6, wherein, The interocular time parallax constraint includes the maximum permissible time offset between the illumination windows.

8. The method according to claim 1, wherein, The lighting subsystem includes an independently controllable backlight.

9. The method according to claim 1, wherein, The lighting subsystem includes segmented lighting sources, which are independently selected for the left eye region and the right eye region.

10. The method according to claim 1, wherein, The lighting windows overlap in time.

11. The method according to claim 1, further comprising: The content adaptively modulates the illumination duty cycle within at least one illumination window.

12. The method of claim 11, further comprising: In response to duty cycle modulation, perceived brightness is compensated by adjusting the effective brightness via at least one of pixel driving or illumination amplitude.

13. The method according to claim 1, further comprising: The motion state of the near-eye display or the user's head posture is sensed, and the emission timing of each region is adjusted based at least in part on the motion state.

14. The method according to claim 1, wherein, Sensing the operating conditions includes reading at least one temperature sensor coupled to the image panel or the illumination subsystem.

15. The method according to claim 1, further comprising: The motion-to-photon delay is reduced by advancing at least one illumination window while maintaining interocular alignment.

16. The method according to claim 1, further comprising: Adjust the data link rate or pipeline scheduling to align the refresh state with the coordinated emission timing.

17. The method according to claim 1, further comprising: Perform color difference correction and coordinate the emission timing of each region to address processing delays.

18. The method according to claim 1, wherein, The single image panel rotates relative to the optical axis, causing the refresh states of the left eye region and the right eye region to proceed in opposite directions.

19. A system comprising: At least one physical processor; Physical memory, the physical memory including computer-executable instructions, which, when executed by the physical processor, cause the physical processor to: A near-eye display is driven, the near-eye display including a single image panel divided into a left-eye region and a right-eye region, and a corresponding independently controllable illumination subsystem; Sensing the operating conditions of the near-eye display; The lighting window for each region is determined at least in part based on the refresh status of each region and the operating conditions. as well as The illumination subsystem is instructed to emit light during the defined illumination window, while coordinating the timing of light emission in each region to achieve inter-eye alignment and reduce motion artifacts caused by the display.

20. A non-transitory computer-readable medium comprising one or more computer-executable instructions, which, when executed by at least one processor of a computing device, cause the computing device to: A near-eye display is driven, the near-eye display including a single image panel divided into a left-eye region and a right-eye region, and a corresponding independently controllable illumination subsystem; Sensing the operating conditions of the near-eye display; The lighting window for each region is determined at least in part based on the refresh status of each region and the operating conditions. as well as The illumination subsystem is instructed to emit light during the defined illumination window, while coordinating the timing of light emission in each region to achieve inter-eye alignment and reduce motion artifacts caused by the display.

Images