Geometric waveguide with zero input clock angle

Abstract

A waveguide for a display system includes a waveguide body extending from an input end to an output end and configured to guide light from the input end to the output end by total internal reflection, a coupling-in mirror configured to direct image light into the waveguide body, a rotating mirror configured to rotate the image light within the waveguide body, a light-bending mirror configured to expand the image light within the waveguide body, and a coupling-out mirror configured to direct the image light out of the waveguide body.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 622,601, filed January 19, 2024, pursuant to 35 USC §119(e). Technical Field

[0002] This invention relates to a display system with waveguides. Summary of the Invention

[0003] According to one aspect of the present invention, a display system is provided, the display system comprising: a waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end by total internal reflection; an input mirror configured to guide the image light into the waveguide body; a rotatable mirror configured to rotate the image light within the waveguide body; a refractive mirror configured to extend the image light within the waveguide body; and an output mirror configured to guide the image light out of the waveguide body.

[0004] Optionally, the orientation of the input field of view of the display system and the orientation of the output field of view of the display system are equivalent.

[0005] Optionally, the orientation of the coupling mirror is the same as the orientation of the coupling mirror.

[0006] Optionally, the tilt angle of the input mirror is equal to the tilt angle of the output mirror.

[0007] Optionally, the clocking angle of the input mirror is equal to the clocking angle of the output mirror.

[0008] Optionally, the orientation of the rotating mirror is the same as that of the refractive mirror.

[0009] Optionally, the tilt angle of the rotating mirror is equal to the tilt angle of the refractive mirror.

[0010] Optionally, the clock angle of the rotating mirror is equal to the clock angle of the refractive mirror.

[0011] Optionally, the rotating mirror has a reflectivity of 90% to 100%.

[0012] Optionally, the rotating mirror includes a single reflective element.

[0013] Optionally, the rotating mirror includes a plurality of mirrors arranged in the mirror area.

[0014] Optionally, the display system also includes a projector for providing the image light to the waveguide body.

[0015] Optionally, the rotating reflector is configured to align the projector's field of view with the user's field of view. Optionally, the display system further includes an alignment prism located between the projector and the waveguide body, wherein the alignment prism is configured to align the projector with at least one of a tilt angle and a clock angle relative to the waveguide body.

[0016] According to another aspect of the present invention, a display system is provided, the display system comprising: a projector configured to provide image light; a waveguide configured to guide the image light from an input end of the waveguide to an output end of the waveguide; an input mirror located near the input end of the waveguide and configured to guide the image light into the waveguide; a rotatable mirror configured to rotate the image light within the waveguide; a refractive mirror configured to extend the image light within the waveguide; and an output mirror located near the output end of the waveguide and configured to guide the image light out of the waveguide.

[0017] Optionally, the orientation of the input field of view of the display system and the orientation of the output field of view of the display system are equivalent.

[0018] Optionally, the rotating mirror includes a single reflective element.

[0019] Optionally, the rotating mirror includes a plurality of mirrors arranged in the mirror area.

[0020] Optionally, the display system further includes an alignment prism located between the projector and the waveguide, wherein the alignment prism is configured to align the projector with at least one of a tilt angle and a clock angle relative to the waveguide.

[0021] According to another aspect of the present invention, a display system is provided, the display system comprising: a waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end by total internal reflection; an input mirror configured to guide the image light into the waveguide body; a rotatable mirror configured to rotate the image light within the waveguide body; a refractive mirror configured to extend the image light within the waveguide body; and an output mirror configured to guide the image light out of the waveguide body, wherein the orientation of the input mirror and the orientation of the output mirror are identical; and the orientation of the rotatable mirror and the orientation of the refractive mirror are identical. Attached Figure Description

[0022] The accompanying drawings illustrate several exemplary embodiments and are part of the specification. The drawings, together with the following description, illustrate and explain various principles of this disclosure.

[0023] Figure 1 The diagram illustrates a three-region geometric waveguide architecture and a four-region geometric waveguide architecture according to some embodiments.

[0024] Figure 2 This is a diagram illustrating the clock angle of the field of view of a projector in a comparative three-region geometric waveguide (GWG) according to certain embodiments.

[0025] Figure 3 This is an illustration of the angular relationship between the field of view and the pupil orientation of a projector for a comparative three-region geometric waveguide (GWG) according to some embodiments.

[0026] Figure 4 The diagram illustrates the angular misalignment between the field of view and the pupil orientation of a projector for a contrasting three-region geometric waveguide (GWG) according to some embodiments.

[0027] Figure 5 The clock angle evolution of image light passing through a four-region geometric waveguide is depicted according to some embodiments.

[0028] Figure 6 The angular alignment between the field of view and the pupil direction of a projector in a four-region geometric waveguide (GWG) according to some embodiments is shown.

[0029] Figure 7 The illustration shows a case where an input prism is incorporated between the light source and the four-region GWG, according to certain embodiments.

[0030] Figure 8 This is an illustration of an example artificial reality system according to some embodiments of the present disclosure.

[0031] Figure 9 This is an illustration of an example artificial reality system with a handheld device according to some embodiments of the present disclosure.

[0032] Figure 10A This is an illustration of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0033] Figure 10B This is an illustration of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0034] Figure 11A This is an illustration of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0035] Figure 11BThis is an illustration of example user interactions within an artificial reality system according to some embodiments of the present disclosure.

[0036] Figure 12 This is an illustration of an example wrist-worn wearable device for an artificial reality system according to some embodiments of the present disclosure.

[0037] Figure 13 This is an illustration of an example wearable artificial reality system according to some embodiments of the present disclosure.

[0038] Figure 14 This is an illustration of an example augmented reality system according to some embodiments of the present disclosure.

[0039] Figure 15A This is an illustration of an example virtual reality system according to some embodiments of the present disclosure.

[0040] Figure 15B for Figure 15A An illustration of another perspective of the virtual reality system shown.

[0041] Figure 16 The diagram illustrates the various system components of an example artificial reality system and a virtual reality system according to some embodiments.

[0042] Throughout the accompanying drawings, the same reference numerals and descriptions denote similar but not necessarily identical elements. While various modifications and alternatives are readily possible with respect to the exemplary embodiments described herein, specific embodiments have been illustrated by way of example in the drawings, and these specific embodiments will be described in detail herein. However, the exemplary embodiments described herein are not intended to be limited to the particular forms disclosed. Rather, this disclosure covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. Detailed Implementation

[0043] Virtual reality (VR) eye-wear and head-mounted devices, as well as augmented reality (AR) eye-wear and head-mounted devices, allow users to experience a variety of events, such as interacting with others in a computer-generated 3D world simulation or viewing data overlaid on a real-world view. For example, information can be overlaid onto the field of view using an optical head-mounted display (OHMD) or embedded wireless glasses with a transparent heads-up display (HUD) or augmented reality (AR) overlay. VR / AR eye-wear and head-mounted devices can be used for a wide range of purposes. For instance, governments can use such devices for military training, medical professionals can use them to simulate surgery, and engineers can use them as design visualization aids.

[0044] Virtual reality devices, virtual reality headsets, augmented reality devices, and augmented reality headsets typically include optical systems with microdisplays and imaging optics. The microdisplays are configured to provide an image to be viewed by directly or indirectly using, for example, micro-OLED displays or by illuminating a liquid crystal-based display (e.g., a liquid crystal on silicon (LCoS) microdisplay).

[0045] Display light can be projected onto a user's eye using a waveguide display system, where light is coupled into a waveguide, transmitted through the waveguide via total internal reflection (TIR), and coupled out upon reaching the user's eye. Imaging optics may include coupling elements and coupling elements (e.g., surface-embossed gratings or mirrors) configured to guide light into and out of the waveguide. Vertical gratings or mirror couplers, for example, may be configured to change the out-of-plane wave vector direction of light to an in-plane waveguide direction, or vice versa, and accordingly guide the light through the waveguide.

[0046] Waveguide optics can be advantageously configured to produce illumination uniformity and a wide field of view (FOV). FOV relates to the angular range of the image that a user can observe, while illumination uniformity can include the uniformity of image light over the extended exit pupil (exit pupil uniformity) and the uniformity of image light within the FOV (angular uniformity). As will be understood, an input coupling grating can determine the angular uniformity and coupling efficiency of the image light. Furthermore, the field of view of an augmented reality waveguide can be highly dependent on the refractive index and geometry of the waveguide medium itself.

[0047] In the exemplary system, light can be presented to the user through pupil expansion achieved using waveguide propagation and pupil replication. In systems with compact and commercially relevant form factors, image light can be manipulated to provide virtual object distances that match real-world scenes along a sufficient field of view (FOV) while maintaining good image sharpness.

[0048] However, achieving appropriate pupil replication density and image fidelity in projected images can be challenging, including the emergence of optical artifacts that may result from waveguide displays designed to present large fields of view. For example, for contrast geometry waveguides with a three-region mirror architecture comprising (i) an input mirror, (ii) a refractive mirror, and (iii) an output mirror, performance quality and user experience may be negatively impacted by geometric issues related to the projector's field-of-view clock angle and the clock angle between the projector's field of view and the pupil orientation. Despite recent advancements, it would be advantageous to provide a waveguide display for generating high-fidelity, large-field-of-view images to support high-quality immersive experiences in virtual reality devices and systems, as well as augmented reality devices and systems.

[0049] According to various embodiments, the waveguide display includes a geometric waveguide with a four-region mirror architecture, comprising (i) an input mirror, (ii) a rotating mirror, (iii) a refractive mirror, and (iv) an output mirror. One or more input mirrors and one or more output mirrors may be aligned in identical or substantially identical orientations, and independently, one or more rotating mirrors and one or more refractive mirrors may be aligned in identical or substantially identical orientations. The rotating mirror may be configured to align the field of view of the projector with the field of view of the pupil. The currently disclosed geometric waveguide can be configured for small field-of-view applications (10° to 50°) (e.g., smart glasses), medium field-of-view applications (50° to 70°) (e.g., AR products), and large field-of-view applications (70° to 100°) (e.g., immersive AR systems).

[0050] The following will refer to Figures 1 to 16 Provides a detailed description of the devices and related methods associated with geometric waveguides. Figures 1 to 7 Related discussions include a description of a geometric waveguide (GWG) with a four-region mirror structure and zero-input clock angle. (And...) Figures 8 to 16 The related discussion refers to exemplary virtual reality and augmented reality devices that may include one or more geometric waveguide architectures as disclosed herein.

[0051] Figure 1 A and Figure 1 B describes a comparative three-region geometric waveguide (GWG) architecture and an illustrative four-region geometric waveguide (GWG) architecture, respectively. Figure 1 The three-region GWG of A includes an in-coupling mirror (IM), a folding mirror (FM), and an out-coupling mirror (OM). Figure 1 The four-region GWG of B includes an input mirror (IM), a high-reflectivity rotating mirror (RM), a refracting mirror (FM), and an output mirror (OM). The rotating mirror can be co-integrated with the refracting mirror.

[0052] Figure 2 The field of view of the GWG display projector and various aspects of its associated clock angle are shown. Figure 3 This illustrates various aspects of the clock angle between the pupil orientation and the projector's field of view (FOV). Reference Figure 2 The clock angle of the projector's FOV can be related to the angle between the input FOV and the output FOV. (Go to...) Figure 3 Although the output field of view from the light source can have a specific orientation (e.g., longitudinal or lateral), the image received by the user's eye from the three-zone GWG is typically rotated relative to the projector's field of view due to the inherent clock angle of the projector's field of view. That is, in order to give the received image the desired orientation (e.g., longitudinal or lateral), the image light output from the light source can be rotated to compensate for the inherent rotational effect of the waveguide.

[0053] Figure 4 The diagram illustrates the angular mismatch between the field of view and pupil orientation of a projector with a contrasting three-region geometric waveguide. In the example shown, the field of view (FOV) can be 40° × 30°, the refracting mirror (FM) can be characterized by a clock angle (β) of approximately 60° and a tilt angle (β') of approximately 90°, and the output mirror (OM) can be characterized by a clock angle (γ) of approximately 185° and a tilt angle (γ') of approximately 240°.

[0054] In a particular embodiment, the four-region geometric waveguide (GWG) includes, in addition to one or more coupling mirrors, one or more refractive mirrors, and one or more coupling mirrors, a rotating mirror. The rotating mirror may include a single reflective element having high (e.g., about 90% to 100%) reflectivity, or may include multiple mirrors arranged in a mirror region.

[0055] like Figure 5As shown, a rotating mirror can be configured to efficiently rotate the image light within the waveguide so that the image light orientation at the pupil is aligned with the orientation of the projector's field of view. In some embodiments, the input and output fields of view have identical orientations, wherein the input and output mirrors are co-aligned.

[0056] For example, such as Figure 5 As shown in the k-space angle evolution, the normal vector of the coupled mirror can be oriented at 180°, the normal vector of the coupled mirror can be oriented at 180°, and the clock angle between the projector's FOV and the pupil orientation can be 90°. When the input and output fields of view have the same orientation, the projector's clock angle is aligned. Therefore, the desired orientation of the image light received by the user (e.g., longitudinal or lateral) can be equivalent to or substantially equivalent to the orientation of the image light output by the projector.

[0057] refer to Figure 6 The realignment of the image light angle between the field of view and the pupil orientation of the projector in a four-region geometric waveguide can simplify the design and improve the shape features of the 2D geometric waveguide. In the example shown, the field of view (FOV) can be 40° × 30°, the refractive mirror (FM) and the rotating mirror (RM) can each be characterized by a clock angle (β) of approximately 60° and a tilt angle (β') of approximately 90°, and the input mirror (IM) and the output mirror (OM) can each be characterized by a clock angle (γ) of approximately 185° and a tilt angle (γ') of approximately 240°.

[0058] refer to Figure 7 According to some embodiments, the four-zone geometric waveguide (GWG) may additionally include an alignment prism. The alignment prism can be configured to align the tilt angle and / or clock angle of the projector relative to any type of waveguide. The alignment prism can be integrated into the projector or fixed to the waveguide between the projector and the coupling mirror.

[0059] Example Implementation Example 1: A display system includes: a waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end via total internal reflection; an input mirror configured to guide the image light into the waveguide body; a rotatable mirror configured to rotate the image light within the waveguide body; a refractive mirror configured to spread the image light within the waveguide body; and an output mirror configured to guide the image light out of the waveguide body.

[0060] Example 2: The display system according to Example 1, wherein the orientation of the input field of view of the display system and the orientation of the output field of view of the display system are equivalent.

[0061] Example 3: A display system according to either Example 1 or Example 2, wherein the orientation of the coupling mirror and the orientation of the coupling mirror are identical.

[0062] Example 4: A display system according to any one of Examples 1 to 3, wherein the tilt angle of the coupled mirror is equal to the tilt angle of the coupled mirror.

[0063] Example 5: A display system according to any one of Examples 1 to 4, wherein the clock angle of the coupled-in mirror is equal to the clock angle of the coupled-out mirror.

[0064] Example 6: A display system according to any one of Examples 1 to 5, wherein the orientation of the rotating mirror and the orientation of the refractive mirror are equivalent.

[0065] Example 7: A display system according to any one of Examples 1 to 6, wherein the tilt angle of the rotating mirror is equal to the tilt angle of the refractive mirror.

[0066] Example 8: A display system according to any one of Examples 1 to 7, wherein the clock angle of the rotating mirror is equal to the clock angle of the refractive mirror.

[0067] Example 9: A display system according to any one of Examples 1 to 8, wherein the rotating reflector has a reflectivity of 90% to 100%.

[0068] Example 10: A display system according to any one of Examples 1 to 9, wherein the rotating mirror includes a single reflective element.

[0069] Example 11: A display system according to any one of Examples 1 to 10, wherein the rotating reflector includes a plurality of reflectors arranged in the reflector area.

[0070] Example 12: A display system according to any one of Examples 1 to 11, the display system further includes a projector for providing the image light to the waveguide body.

[0071] Example 13: A display system according to any of Example 12, wherein the rotating reflector is configured to align the field of view of the projector with the field of view of the user.

[0072] Example 14: A display system according to any one of Examples 12 and 13, the display system further includes an alignment prism located between the projector and the waveguide body, wherein the alignment prism is configured to align the projector with at least one of a tilt angle and a clock angle relative to the waveguide body.

[0073] Example 15: A display system includes: a projector configured to provide image light; a waveguide configured to guide the image light from an input end of the waveguide to an output end of the waveguide; an input mirror located near the input end of the waveguide and configured to guide the image light into the waveguide; a rotatable mirror configured to rotate the image light within the waveguide; a refractive mirror configured to extend the image light within the waveguide; and an output mirror located near the output end of the waveguide and configured to guide the image light out of the waveguide.

[0074] Example 16: A display system according to Example 15, wherein the orientation of the input field of view of the display system and the orientation of the output field of view of the display system are equivalent.

[0075] Example 17: A display system according to either Example 15 or Example 16, wherein the rotating mirror includes a single reflective element.

[0076] Example 18: A display system according to any one of Examples 15 to 17, wherein the rotating reflector includes a plurality of reflectors arranged in the reflector area.

[0077] Example 19: A display system according to any one of Examples 15 to 18, the display system further includes an alignment prism located between the projector and the waveguide, wherein the alignment prism is configured to align the projector with at least one of a tilt angle and a clock angle relative to the waveguide.

[0078] Example 20: A display system includes: a waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end via total internal reflection; an input mirror configured to guide the image light into the waveguide body; a rotatable mirror configured to rotate the image light within the waveguide body; a refractive mirror configured to spread the image light within the waveguide body; and an output mirror configured to guide the image light out of the waveguide body, wherein the orientation of the input mirror and the orientation of the output mirror are identical; and the orientation of the rotatable mirror and the orientation of the refractive mirror are identical.

[0079] Embodiments of this disclosure may include various types of artificial reality (AR) systems or combinations thereof. AR can be any overlay of functionality and / or sensorially detectable content presented by an AR system within the user's physical environment. In other words, AR is a form of reality that has been modulated in some way before being presented to the user. AR may include and / or represent virtual reality (VR), augmented reality, mixed AR (MAR), or some combination and / or variation of these types of reality. Similarly, AR environments may include VR environments (including non-immersive VR environments, semi-immersive VR environments, and fully immersive VR environments), augmented reality environments (including marked augmented reality environments, unmarked augmented reality environments, location-based augmented reality environments, and projection-based augmented reality environments), mixed reality environments, and / or any other type or form of mixed reality environment or alternative reality environment.

[0080] AR content can include entirely computer-generated content or computer-generated content combined with acquired (e.g., real-world) content. Such AR content can include video, audio, haptic feedback, or some combination thereof, any of which can be presented in a single channel or multiple channels (e.g., stereoscopic video that produces a three-dimensional (3D) effect for the viewer). Furthermore, in some embodiments, AR can also be associated with applications, products, accessories, services, or some combination thereof for purposes such as creating content in artificial reality and / or otherwise using it in artificial reality (e.g., performing activities in artificial reality).

[0081] AR systems can be implemented in a variety of different shapes and configurations. Some AR systems can be designed to operate without a near-eye display (NED). Other AR systems may include NEDs that also provide visibility into the real world (e.g., Figure 14 Augmented reality systems (1400) or NEDs that visually immerse users in artificial reality (e.g., Figure 15A and Figure 15B (In the context of virtual reality systems). While some AR devices can be standalone systems, others can communicate and / or coordinate with external devices to provide an AR experience to the user. Examples of such external devices include handheld controllers, mobile devices, desktop computers, wearable devices, one or more other wearable devices, and / or any other suitable external systems.

[0082] Figures 8 to 11B An example artificial reality (AR) system according to some embodiments is shown. Figure 8The first example user interaction is shown with the first AR system 800 and using a wrist wearable device 802, a head wearable device (e.g., AR glasses 1400) and / or a handheld intermediary processing device (HIPD) 806. Figure 9 The second AR system 900 and a second example user interaction using a wrist-worn wearable device 902, AR glasses 904 and / or HIPD 906 are shown. Figure 10A and Figure 10B The interaction between the third AR system 1000 and a third example user 1008 using a wrist wearable device 1002, a head wearable device (e.g., a VR headset 1050), and / or a HIPD 1006 is shown. Figure 11A and Figure 11B The fourth AR system 1100 is shown interacting with a fourth example user 1108 using a wrist wearable device 1130, a VR headset 1120, and / or a haptic device 1160 (e.g., wearable gloves).

[0083] The wrist wearable device 1200 and one or more components thereof, which can be used in wrist wearable devices 802, 902, 1002, and 1130, are referred to below. Figure 12 and Figure 13 Described below; head-worn wearable devices 1400 and 1500, which can be used respectively for AR glasses 804, 904 or VR headsets 1050, 1120, and one or more of their components are referenced. Figures 14 to 16 Describe it.

[0084] refer to Figure 8 The wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can be communicatively coupled via a network 825 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN, etc.). Additionally, the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can also be communicatively coupled via the network 825 (e.g., cellular, near-field, Wi-Fi, personal area network, wireless LAN, etc.) to one or more servers 830, computers 840 (e.g., laptops, computers, etc.), mobile devices 850 (e.g., smartphones, tablets, etc.), and / or other electronic devices.

[0085] exist Figure 8The image shows a user 808 wearing a wrist-worn wearable device 802 and AR glasses 804, with a HIPD 806 placed on their table. The wrist-worn wearable device 802, AR glasses 804, and HIPD 806 facilitate user interaction with the AR environment. Specifically, as shown in the first AR system 800, the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 enable the presentation of one or more avatars 810, digital representations of contacts 812, and virtual objects 814. As described below, the user 808 can interact with one or more avatars 810, digital representations of contacts 812, and virtual objects 814 via the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806.

[0086] User 808 may use any of the wrist wearable device 802, AR glasses 804, and / or HIPD 806 to provide user input. For example, user 808 may perform actions by the wrist wearable device 802 (e.g., using one or more EMG sensors and / or IMU, referred to below). Figure 12 and Figure 13 (Description) and / or AR glasses 804 (e.g., using one or more image sensors or cameras, see below for reference) Figures 14 to 16 (Description) Detects one or more gestures to provide user input. Alternatively or additionally, user 808 may provide user input via one or more touch surfaces of the wrist wearable device 802, AR glasses 804, HIPD 806, and / or voice commands collected by the microphones of the wrist wearable device 802, AR glasses 804, and / or HIPD 806. In some embodiments, the wrist wearable device 802, AR glasses 804, and / or HIPD 806 includes a digital assistant to assist user 808 in providing user input (e.g., performing a series of actions, suggesting different actions or commands, providing reminders, confirming commands, etc.). In some embodiments, user 808 may provide user input via one or more facial gestures and / or facial expressions. For example, the camera of the wrist wearable device 802, AR glasses 804, and / or HIPD 806 may track user 808's eyes to navigate the user interface.

[0087] The wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 can operate individually or in combination to allow user 808 to interact with the AR environment. In some embodiments, HIPD 806 is configured to operate as a central hub or control center for the wrist-worn wearable device 802, AR glasses 804, and / or another communicatively coupled device. For example, user 808 can provide input at any of the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806 to interact with the AR environment, and HIPD 806 can identify one or more backend and frontend tasks to cause the requested interaction to be performed, and distribute instructions to cause one or more backend and frontend tasks to be performed at the wrist-worn wearable device 802, AR glasses 804, and / or HIPD 806. In some embodiments, backend tasks are background processing tasks imperceptible to the user (e.g., rendering content, decompressing, compressing, etc.), and frontend tasks are user-perceptible, user-facing tasks (e.g., presenting information to the user, providing feedback to the user, etc.). HIPD 806 can perform backend tasks and provide operational data corresponding to the backend tasks to the wrist wearable device 802 and / or AR glasses 804, enabling the wrist wearable device 802 and / or AR glasses 804 to perform frontend tasks. In this way, HIPD 806, which has more computing resources and a larger thermal headroom than the wrist wearable device 802 and / or AR glasses 804, performs computationally intensive tasks and reduces the computing resource utilization and / or power consumption of the wrist wearable device 802 and / or AR glasses 804.

[0088] In the example shown in the first AR system 800, HIPD 806 identifies one or more backend and frontend tasks associated with a user request initiating an AR video call with one or more other users (represented by avatar 810 and digital representation 812 of contacts), and issues instructions to cause the execution of one or more backend and frontend tasks. Specifically, HIPD 806 performs backend tasks for processing and / or rendering image data (and other data) associated with the AR video call, and provides AR glasses 804 with operational data associated with the performed backend tasks, causing AR glasses 804 to perform frontend tasks for presenting the AR video call (e.g., presenting avatar 810 and digital representation 812 of contacts).

[0089] In some embodiments, HIPD 806 can be used as a focal point or anchor point for presenting information. This allows user 808 to generally know where the information is presented. For example, as shown in the first AR system 800, avatar 810 and contact digital representations 812 are presented above HIPD 806. Specifically, HIPD 806 and AR glasses 804 operate in conjunction to determine the location for presenting avatar 810 and contact digital representations 812. In some embodiments, information can be presented at a predetermined distance from HIPD 806 (e.g., within 5 meters). For example, as shown in the first AR system 800, virtual object 814 is presented on a table at a distance from HIPD 806. Similar to the above examples, HIPD 806 and AR glasses 804 can operate in conjunction to determine the location for presenting virtual object 814. Alternatively, in some embodiments, the presentation of information is not constrained by HIPD 806. More specifically, avatar 810, contact digital representations 812, and virtual object 814 do not necessarily need to be presented within the predetermined distance of HIPD 806.

[0090] User input provided at the wrist wearable device 802, AR glasses 804, and / or HIPD 806 is coordinated to enable the user to initiate, continue, and / or complete an operation using any device. For example, user 808 may provide user input to AR glasses 804 to cause AR glasses 804 to present a virtual object 814, and when AR glasses 804 presents the virtual object 814, user 808 may provide one or more gestures via the wrist wearable device 802 to interact with and / or manipulate the virtual object 814.

[0091] Figure 9 The illustration shows a user 908 wearing a wrist-worn wearable device 902 and AR glasses 904, and holding a HIPD 906. In the second AR system 900, the wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906 are used to receive one or more messages and / or provide one or more messages to the user 908's contacts. Specifically, the wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906 detect and coordinate one or more user inputs to initiate a messaging application and prepare responses to messages received via the messaging application.

[0092] In some embodiments, user 908 initiates an application on a wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906 via user input, causing the application to be launched on at least one device. For example, in a second AR system 900, user 908 performs a gesture associated with a command (represented by a messaging user interface 916) to launch a messaging application. The wrist-worn wearable device 902 detects the gesture and, based on determining that user 908 is wearing AR glasses 904, causes AR glasses 904 to present the messaging user interface 916 of the messaging application. AR glasses 904 may present the messaging user interface 916 to user 908 via its display (e.g., as shown in user 908's field of view 918). In some embodiments, the application is launched and executed on a device (e.g., wrist-worn wearable device 902, AR glasses 904, and / or HIPD 906) that detects user input to launch the application, and that device provides operational data to another device to cause the messaging application to be presented. For example, the wrist-worn wearable device 902 can detect user input to launch a messaging application, start and run the messaging application, and provide operational data to the AR glasses 904 and / or HIPD 906 to enable the presentation of the messaging application. Alternatively, the application can be launched and executed on a device other than the one that detects user input. For example, the wrist-worn wearable device 902 can detect gestures associated with launching the messaging application and enable the HIPD 906 to run the messaging application and coordinate its presentation.

[0093] Furthermore, user 908 can provide user input at the wrist wearable device 902, AR glasses 904, and / or HIPD 906 to continue and / or complete an operation initiated at another device. For example, after launching a messaging application via the wrist wearable device 902, and when the AR glasses 904 presents the messaging user interface 916, user 908 can provide input at HIPD 906 to prepare a response (e.g., indicated by a swipe gesture performed on HIPD 906). Gestures performed by user 908 on HIPD 906 can be provided and / or displayed on another device. For example, a swipe gesture performed on HIPD 906 is displayed on the virtual keyboard of the messaging user interface 916 displayed by AR glasses 904.

[0094] In some embodiments, the wrist wearable device 902, AR glasses 904, HIPD 906, and / or any other communication-coupled device may present one or more notifications to the user 908. The notification may be an indication of a new message, incoming call, application update, status update, etc. The user 908 may select a notification via the wrist wearable device 902, AR glasses 904, and / or HIPD 906, and may cause an application or action associated with the notification to be presented on at least one device. For example, the user 908 may receive a notification that a message has been received at the wrist wearable device 902, AR glasses 904, HIPD 906, and / or any other communication-coupled device, and may then provide user input at the wrist wearable device 902, AR glasses 904, and / or HIPD 906 to view the notification. The device that detects the user input may cause an application associated with the notification to be launched and / or presented at the wrist wearable device 902, AR glasses 904, and / or HIPD 906.

[0095] While the examples above describe coordinated input for interacting with messaging applications, user input can be coordinated to interact with any number of applications, including but not limited to gaming applications, social media applications, camera applications, web-based applications, financial applications, etc. For example, AR glasses 904 can present game application data to user 908, and HIPD 906 can be used as a controller to provide input to the game. Similarly, user 908 can use wrist wearable device 902 to activate the camera of AR glasses 904, and user 908 can use wrist wearable device 902, AR glasses 904, and / or HIPD 906 to manipulate image acquisition (e.g., zoom in or out, apply filters, etc.) and acquire image data.

[0096] Users can interact with the devices disclosed in this article in a variety of ways. For example, such as Figure 10A and Figure 10B As shown, user 1008 can interact with AR system 1000 by wearing VR headset 1050, holding HIPD 1006, and wearing wrist wearable device 1002. In this example, AR system 1000 allows the user to interact with game 1010 by waving their arm. One or more of VR headset 1050, HIPD 1006, and wrist wearable device 1002 can detect the gesture and, in response, can display sword strikes in game 1010. Similarly, in Figure 11A and Figure 11BIn this example, user 1108 can interact with AR system 1100 by wearing VR headset 1120 while simultaneously wearing haptic device 1160 and wrist wearable device 1130. In this example, AR system 1100 allows the user to interact with game 1110 by waving their arm. One or more of the VR headset 1120, haptic device 1160, and wrist wearable device 1130 can detect the gesture and, in response, can display a fireball being thrown in game 1010.

[0097] Having discussed example AR systems, this paper will now discuss in more detail the devices used to interact with such AR systems and other computing systems. For ease of reference, this paper explains some of the devices and components that may be included in some or all of the example devices discussed below. Certain types of components described below may be more suitable for a particular set of devices and less suitable for different sets of devices; however, subsequent references to components explained herein should be considered to be covered by the descriptions provided.

[0098] In some embodiments discussed below, example devices and systems, including electronic devices and systems, will be described. Such example devices and systems are not intended to be limiting, and those skilled in the art will understand that alternative devices and systems to the example devices and systems described herein can be used to perform operations and construct the systems and devices described herein.

[0099] An electronic device can be a device that uses electrical energy to perform a specific function. An electronic device can be any physical object that contains electronic components such as transistors, resistors, capacitors, diodes, and integrated circuits. Examples of electronic devices include smartphones, laptops, digital cameras, televisions, game consoles, and music players, as well as the example electronic devices discussed herein. As described herein, an intermediate electronic device can be a device located between two other electronic devices and / or subsets of components of one or more electronic devices, and facilitates communication, data processing, and / or data transmission between the various electronic devices and / or electronic components.

[0100] An integrated circuit can be an electronic device composed of multiple interconnected electronic components, such as transistors, resistors, and capacitors. These components can be etched onto small pieces of semiconductor material, such as silicon. Integrated circuits can include analog integrated circuits, digital integrated circuits, mixed-signal integrated circuits, and / or any other suitable type or form of integrated circuit. Examples of integrated circuits include application-specific integrated circuits (ASICs), processing units, central processing units (CPUs), coprocessors, and accelerators.

[0101] Analog integrated circuits (such as sensors, power management circuits, and operational amplifiers) can process continuous signals and perform analog functions (such as amplification, active filtering, demodulation, and mixing). Examples of analog integrated circuits include linear integrated circuits and radio frequency circuits.

[0102] Digital integrated circuits (which may be referred to as logic integrated circuits) may include microprocessors, microcontrollers, memory chips, interfaces, power management circuits, programmable devices, and / or any other suitable type or form of integrated circuit. In some embodiments, an example of an integrated circuit includes a central processing unit (CPU).

[0103] A processing unit (e.g., a CPU) can be an electronic component responsible for executing instructions and controlling the operation of an electronic device (e.g., a computer). Various types of processors exist, which can be used interchangeably or may be particularly needed in the embodiments described herein. For example, a processor can be: (i) a general-purpose processor designed to perform a wide range of tasks, such as running software applications, managing operating systems, and performing arithmetic and logical operations; (ii) a microcontroller designed for specific tasks, such as controlling electronic devices, sensors, and motors; (iii) an accelerator, such as a graphics processing unit (GPU) designed to accelerate the creation and rendering of images, videos, and animations (e.g., virtual reality animation, such as 3D modeling); (iv) a field-programmable gate array (FPGA) that can be programmed and reconfigured post-manufacturing and / or customized to perform specific tasks, such as signal processing, cryptography, and machine learning; and / or (v) a digital signal processor (DSP) designed to perform mathematical operations on signals such as audio, video, and radio waves. One or more processors of one or more electronic devices may be used in the various embodiments described herein.

[0104] Memory generally refers to electronic components in a computer or electronic device that store data and instructions for access and manipulation by a processor. Examples of memory may include: (i) random access memory (RAM) configured to temporarily store data and instructions; (ii) read-only memory (ROM) configured to permanently store data and instructions (e.g., one or more portions of system firmware, and / or bootloaders) and / or semi-permanently store data and instructions; (iii) flash memory (e.g., USB drives, memory cards, and / or solid-state drives (SSDs)) configured to store data in an electronic device; and / or (iv) cache memory configured to temporarily store frequently accessed data and instructions. As described herein, memory may store structured data (e.g., SQL databases, MongoDB databases, GraphQL data, JSON data, etc.). Other examples of data stored in memory may include: (i) data data, including user account data, user settings and / or other user data stored by the user; (ii) sensor data detected by one or more sensors and / or otherwise acquired; (iii) media content data, including stored image data, audio data and documents, etc.; and (iv) application data, which may include data collected and / or otherwise acquired and stored during use of the application, and / or any other types of data described herein.

[0105] A controller can be an electronic component that manages and coordinates the operation of other components within an electronic device (e.g., controlling inputs, processing data, and / or generating outputs). Examples of controllers may include: (i) microcontrollers, including small, low-power controllers commonly used in embedded systems and Internet of Things (IoT) devices; (ii) programmable logic controllers (PLCs) that can be configured for use in industrial automation systems to control and monitor manufacturing processes; (iii) system-on-a-chip (SoC) controllers that integrate multiple components such as processors, memory, I / O interfaces, and other peripherals onto a single chip; and / or (iv) DSPs.

[0106] The power system of an electronic device can be configured to convert input electrical energy into a form usable for operating the device. The power system may include various components, such as (i) a power supply, which may be an alternating current (AC) adapter power supply or a direct current (DC) adapter power supply; (ii) a charger input, which may be configured to use wired and / or wireless connections (which may be part of a peripheral interface, such as a universal serial bus (USB), a micro USB interface, near-field magnetic coupling, magnetic induction and magnetic resonance charging, and / or radio frequency (RF) charging); (iii) a power management integrated circuit configured to distribute power to various components of the device and ensure that the device operates within safe limits (e.g., regulating voltage, controlling current, and / or managing heat dissipation); and / or (iv) a battery configured to store power to provide usable power to components of one or more electronic devices.

[0107] Peripheral interfaces can be (e.g., electronic components of an electronic device) that allow the electronic device to communicate with other devices or peripheral devices and can provide the ability to input and output data and signals. Examples of peripheral interfaces may include: (i) a Universal Serial Bus (USB) and / or a micro USB interface configured to connect a device to an electronic device; (ii) a Bluetooth interface configured to allow devices to communicate with each other, including Bluetooth Low Energy (BLE); (iii) a Near Field Communication (NFC) interface configured as a short-range wireless interface for operations such as access control; (iv) a POGO pin, which can be a small, spring-loaded pin configured to provide a charging interface; (v) a wireless charging interface; (vi) a GPS interface; (vii) a Wi-Fi interface used to provide connectivity between the device and a wireless network; and / or (viii) a sensor interface.

[0108] Sensors can be electronic components (e.g., electronic components in or otherwise communicating electronically with electronic devices, such as wearable devices) configured to detect physical and environmental changes and generate electrical signals. Examples of sensors may include (i) imaging sensors for collecting imaging data (e.g., including one or more cameras mounted on a corresponding electronic device), (ii) biopotential signal sensors, (iii) inertial measurement units (e.g., multiple IMUs) for detecting changes in, for example, angular velocity, force, magnetic field, and / or acceleration, (iv) heart rate sensors for measuring a user's heart rate, (v) SpO2 sensors for measuring a user's blood oxygen saturation and / or other biometric data, (vi) capacitive sensors for detecting potential changes in a part of the user's body (e.g., a sensor-skin interface), and / or (vii) light sensors (e.g., time-of-flight sensors, infrared sensors, visible light sensors, etc.).

[0109] Biopotential signal sensing components can be devices used to measure electrical activity within the body (e.g., biopotential signal sensors). Some types of biopotential signal sensors include: (i) electroencephalography (EEG) sensors, which are configured to measure electrical activity in the brain to diagnose neurological disorders; (ii) electrocardiography (ECG or EKG) sensors, which are configured to measure electrical activity in the heart to diagnose heart problems; (iii) electromyography (EMG) sensors, which are configured to measure electrical activity in muscles and diagnose neuromuscular disorders; and (iv) electrooculography (EOG) sensors, which are configured to measure electrical activity in the eye muscles to detect eye movements and diagnose eye disorders.

[0110] Applications (e.g., software) stored in the memory of an electronic device may include instructions stored in the memory. Examples of such applications include: (i) games; (ii) word processors; (iii) messaging applications; (iv) media streaming applications; (v) financial applications; (vi) calendars; (vii) clocks; and (viii) communication interface modules for enabling wired and / or wireless connections between different corresponding electronic devices (e.g., IEEE 1402.15.4, Wi-Fi, ZigBee, 6LoWPAN, Thread, Z-Wave, Bluetooth Smart, ISA100.11a, WirelessHART, or MiWi; custom wired protocols or standard wired protocols (e.g., Ethernet or HomePlug); and / or any other suitable communication protocol).

[0111] A communication interface can be a mechanism that enables different systems or devices to exchange information and data with each other, including hardware, software, or a combination of both. For example, a communication interface can refer to a physical connector and / or port on a device that enables communication with other devices (e.g., USB, Ethernet, High Definition Multimedia Interface (HDMI), Bluetooth). In some embodiments, a communication interface can refer to a software layer that enables different software programs to communicate with each other (e.g., application programming interface (API), protocols such as Hypertext Transfer Protocol (HTTP) and Transmission Control Protocol / Internet Protocol (TCP / IP)).

[0112] A graphics module can be a component or software module designed to handle graphics operations and / or graphical processes, and the graphics module may include hardware modules and / or software modules.

[0113] Non-transitory computer-readable storage media can be physical devices or storage media that can be used to store electronic data in a non-transitory form (e.g., such that the data is permanently stored until it is intentionally deleted or modified).

[0114] Figure 12 and Figure 13 An example wrist-worn wearable device 1200 and an example computing system 1300 are illustrated according to some embodiments. The wrist-worn wearable device 1200 is described herein... Figure 8 The wearable device 802 described herein is such that wearable device 802 should be understood as having the characteristics of wrist wearable device 1200, and vice versa. Figure 13Multiple components of a wrist-worn wearable device 1200 are shown, which can be used individually or in combination, including combinations that include other electronic devices and / or electronic components.

[0115] As discussed below, Figure 12 A wearable strap 1210 and a watch body 1220 (or capsule) are shown coupled to form a wrist-worn wearable device 1200. The wrist-worn wearable device 1200 can perform various functions and / or operations associated with navigating in a user interface and selectively activating applications, as well as those described above. Figures 8 to 11B The described functions and / or operations.

[0116] As will be described in more detail below, the operations performed by the wrist-worn wearable device 1200 may include: (i) presenting content to a user (e.g., displaying visual content via display 1205); (ii) detecting (e.g., sensing) user input (e.g., sensing touches on peripheral buttons 1223 and / or touches on the touchscreen of display 1205, sensing gestures detected by sensors (e.g., biopotential sensors); (iii) sensing biometric data (e.g., neuromuscular signals, heart rate, temperature, sleep, etc.) via one or more sensors 1213; sending and receiving messages (e.g., text, voice, video, etc.); image acquisition via one or more imaging devices or cameras 1225; wireless communication (e.g., cellular, near-field, Wi-Fi, personal area network, etc.); location determination; financial transactions; providing haptic feedback; providing warnings; providing notifications; providing biometric authentication; providing health monitoring; providing sleep monitoring, etc.

[0117] The example functions described above can be performed independently in the watch body 1220, independently in the wearable band 1210, and / or via electronic communication between the watch body 1220 and the wearable band 1210. In some embodiments, the functions can be performed on the wrist wearable device 1200 when an AR environment is presented (e.g., via one of the AR systems 800 to 1100). The wearable device described herein can also be used with other types of AR environments.

[0118] The wearable band 1210 can be configured to be worn by a user such that the inner surface of the wearable structure 1211 of the wearable band 1210 contacts the user's skin. In this example, the sensor 1213 can contact the user's skin when worn by the user. In some examples, one or more of these sensors 1213 can sense biometric data, such as the user's heart rate, saturated oxygen level, temperature, sweat level, neuromuscular signals, or combinations thereof. One or more of these sensors 1213 can also sense data about the user's environment, including the user's movement, height, location, orientation, gait, acceleration, position, or combinations thereof. In some embodiments, one or more of these sensors 1213 can be configured to track the position and / or movement of the wearable band 1210. One or more of these sensors 1213 may include the features defined above and / or the following regarding... Figure 12 Any of the multiple sensors discussed.

[0119] One or more of these sensors 1213 may be distributed on the inner and / or outer surface of the wearable band 1210. In some embodiments, one or more of these sensors 1213 are evenly spaced along the wearable band 1210. Alternatively, in some embodiments, one or more of these sensors 1213 are located at different points along the wearable band 1210. Figure 12 As shown, one or more of these sensors 1213 may be the same or different. For example, in some embodiments, one or more of these sensors 1213 may be shaped as a pill (e.g., sensor 1213a), oval, circular, square, elliptical (e.g., sensor 1213c), and / or any other shape that maintains contact with the user's skin (e.g., so that neuromuscular signals and / or other biometric data can be accurately measured at the user's skin). In some embodiments, one or more of these sensors 1213 are aligned to form sensor pairs (e.g., for sensing neuromuscular signals based on differential sensing within each respective sensor). For example, sensor 1213b may be aligned with an adjacent sensor to form sensor pair 1214a, and sensor 1213d may be aligned with an adjacent sensor to form sensor pair 1214b. In some embodiments, the wearable band 1210 does not have sensor pairs. Alternatively, in some embodiments, the wearable band 1210 has a predetermined number of sensor pairs (one sensor pair, three sensor pairs, four sensor pairs, six sensor pairs, sixteen sensor pairs, etc.).

[0120] The wearable band 1210 may include any suitable number of sensors 1213. In some embodiments, the number and arrangement of the sensors 1213 depend on the specific application using the wearable band 1210. For example, the wearable band 1210 may be configured as an armband, wristband, or chest band including a plurality of sensors 1213, wherein each use case (e.g., a medical use case compared to a gaming use case or a general daily use case) has a different number of sensors 1213, multiple types of individual sensors among the plurality of sensors 1213, and different arrangements.

[0121] According to some embodiments, the wearable band 1210 also includes an electrically grounding electrode and a shielding electrode. Similar to sensor 1213, the electrically grounding electrode and shielding electrode may be distributed on the inner surface of the wearable band 1210 such that they contact a portion of the user's skin. For example, the electrically grounding electrode and shielding electrode may be located on the inner surface of coupling mechanism 1216 or on the inner surface of wearable structure 1211. The electrically grounding electrode and shielding electrode may be formed of and / or use the same components as sensor 1213. In some embodiments, the wearable band 1210 includes more than one electrically grounding electrode and more than one shielding electrode.

[0122] Sensor 1213 may be formed as part of the wearable structure 1211 of the wearable band 1210. In some embodiments, sensor 1213 is flush or substantially flush with the wearable structure 1211, such that these sensors do not extend beyond the surface of the wearable structure 1211. Although flush with the wearable structure 1211, sensor 1213 is still configured to contact the user's skin (e.g., via a skin contact surface). Alternatively, in some embodiments, sensor 1213 extends beyond the wearable structure 1211 by a predetermined distance (e.g., 0.1 mm to 2 mm) to contact and press into the user's skin. In some embodiments, sensor 1213 is coupled to an actuator (not shown) configured to adjust the extension height of sensor 1213 (e.g., distance from the surface of the wearable structure 1211) such that sensor 1213 contacts and presses into the user's skin. In some embodiments, the actuator adjusts the extension height between 0.01 mm and 1.2 mm. This allows users to customize the positioning of sensor 1213 to improve the overall comfort of the wearable band 1210 when worn, while still allowing sensor 1213 to contact the user's skin. In some embodiments, sensor 1213 is not distinguishable from wearable structure 1211 when worn by the user.

[0123] The wearable structure 1211 may be formed of an elastic material, elastomer, or the like configured to be stretched and suitable for being worn by a user. In some embodiments, the wearable structure 1211 is a textile or woven fabric. As described above, the sensor 1213 may be formed as part of the wearable structure 1211. For example, the sensor 1213 may be molded into the wearable structure 1211, integrated into the woven fabric (e.g., the sensor 1213 may be sewn into the fabric and mimic the flexibility of the fabric, and may and / or may be composed of a series of woven fabric threads).

[0124] Wearable structure 1211 may include sensors 1213, electronic circuitry and / or other electronic components (hereinafter referred to as...) to be included in wearable band 1210. Figure 13 (As described above) Flexible electronic connectors interconnect. In some embodiments, the flexible electronic connectors are configured to interconnect sensors 1213, electronic circuitry, and / or other electronic components of the wearable band 1210 with corresponding sensors and / or other electronic components of another electronic device (e.g., the watch body 1220). The flexible electronic connectors are configured to move together with the wearable structure 1211 such that adjustments made to the wearable structure 1211 by the user (e.g., adjusting size, pulling, folding, etc.) do not cause stress or strain on the electrical coupling of the components of the wearable band 1210.

[0125] As described above, the wearable band 1210 is configured to be worn by a user. Specifically, the wearable band 1210 may be shaped or otherwise manipulated for wear by a user. For example, the wearable band 1210 may be shaped to have a generally circular shape, such that the wearable band can be configured to be worn on the user's forearm or wrist. Alternatively, the wearable band 1210 may be shaped to be worn on another body part of the user (e.g., the user's upper arm (e.g., around the biceps), forearm, chest, leg, etc.). The wearable band 1210 may include a retaining mechanism 1212 (e.g., a hook and loop fastener, etc.) for securing the wearable band 1210 to the user's wrist or other body part. When the wearable band 1210 is worn by the user, the sensor 1213 senses data from the user's skin (referred to as sensor data). In some examples, the sensor 1213 of the wearable band 1210 acquires (e.g., senses and records) neuromuscular signals.

[0126] Sensed data (e.g., sensed neuromuscular signals) can be used to detect and / or determine a user's intention to perform certain motor actions. In some examples, sensor 1213 can sense and record the user's neuromuscular signals when the user performs muscle activation (e.g., movement, gesture, etc.). Detected and / or determined motor actions (e.g., phalanges (or fingers) movement, wrist movement, hand movement, and / or other muscle intentions) can be used to determine control commands or control information (instructions to execute certain commands after the data is sensed) for causing the computing device to execute one or more input commands. For example, sensed neuromuscular signals can be used to control certain user interfaces displayed on display 1205 of the wrist-worn wearable device 1200, and / or can be sent to a device responsible for rendering an artificial reality environment (e.g., a head-mounted display) to perform actions in the associated artificial reality environment (e.g., to control the movement of a virtual device displayed to the user). Muscle activation performed by a user can include: static gestures, such as placing the user's palm down on a table; dynamic gestures, such as grasping a physical or virtual object; and covert gestures that are imperceptible to another person, such as slightly tensing a joint by coordinating the contraction of opposing muscles or using submuscular activation. Muscle activation performed by a user can also include symbolic gestures (e.g., gestures mapped to other gestures, interactions, or commands based on a gesture vocabulary that specifies a mapping from gestures to commands).

[0127] The sensor data sensed by sensor 1213 can be used to provide users with enhanced interaction with physical objects (e.g., devices communicatively coupled to wearable strap 1210) and / or virtual objects generated by artificial reality systems in artificial reality applications (e.g., user interface objects presented on display 1205 or another computing device (e.g., smartphone)).

[0128] In some embodiments, the wearable band 1210 includes one or more tactile devices 1346 (e.g., vibratory tactile actuators) configured to provide tactile feedback (e.g., skin sensation and / or kinesthetic perception, etc.) to the user's skin. Sensors 1213 and / or tactile devices 1346 (in...) Figure 13 (As shown in the figure) can be configured to run in combination with multiple applications, including but not limited to health monitoring, social media, games and artificial reality (e.g., applications associated with artificial reality).

[0129] The wearable strap 1210 may also include a coupling mechanism 1216 for detachably coupling a compartment (e.g., a computing unit) or the watch body 1220 to the wearable strap 1210 (via a coupling surface of the watch body 1220). For example, the bracket or shape of the coupling mechanism 1216 may correspond to the shape of the watch body 1220 of the wrist wearable device 1200. In particular, the coupling mechanism 1216 may be configured to receive a coupling surface of the watch body 1220 near the bottom side (e.g., the side opposite the front side where the display 1205 of the watch body 1220 is located) so that a user can push the watch body 1220 down into the coupling mechanism 1216 to attach the watch body 1220 to the coupling mechanism 1216. In some embodiments, the coupling mechanism 1216 may be configured to receive the top side of the watch body 1220 (e.g., the side near the front of the display 1205 of the watch body 1220) which is pushed upward into the bracket rather than downward into the coupling mechanism 1216. In some embodiments, the coupling mechanism 1216 is an integrated component of the wearable strap 1210, such that the wearable strap 1210 and the coupling mechanism 1216 are a single unified structure. In some embodiments, the coupling mechanism 1216 is a frame or housing that allows the coupling surface of the watch body 1220 to remain within or on the coupling mechanism 1216 of the wearable strap 1210 (e.g., bracket, tracking strap, support base, buckle, etc.).

[0130] The coupling mechanism 1216 allows the watch body 1220 to be detachably coupled to the wearable strap 1210 via friction engagement, magnetic coupling, rotation-based connectors, shear pin couplings, retaining springs, one or more magnets, clips, pins, hook-and-loop fasteners, or combinations thereof. A user can perform any type of action to couple the watch body 1220 to and detach it from the wearable strap 1210. For example, a user can twist, slide, rotate, push, pull, or rotate (or combinations thereof) the watch body 1220 relative to the wearable strap 1210 to attach and detach it from the wearable strap 1210. Alternatively, as discussed below, in some embodiments, the watch body 1220 can be detached from the wearable strap 1210 by actuation of the release mechanism 1229.

[0131] The wearable strap 1210 can be coupled to the watch body 1220 to enhance the functionality of the wearable strap 1210 (e.g., converting the wearable strap 1210 into a wrist-worn wearable device 1200, adding additional computing units and / or batteries to increase the computing resources and / or battery life of the wearable strap 1210, adding additional sensors to improve sensed data, etc.). As described above, the wearable strap 1210 and coupling mechanism 1216 are configured to operate independently of the watch body 1220 (e.g., perform functions independently of the watch body). For example, the coupling mechanism 1216 may include one or more sensors 1213 that contact the user's skin with or without the watch body 1220 when the user is wearing the wearable strap 1210, and may provide sensor data for determining control commands.

[0132] Users can detach the watch body 1220 from the wearable strap 1210 to reduce the burden of the wrist wearable device 1200 on the user. In embodiments where the watch body 1220 is detachable, the watch body 1220 may be referred to as a detachable structure, such that in these embodiments, the wrist wearable device 1200 includes a wearable portion (e.g., the wearable strap 1210) and a detachable structure (e.g., the watch body 1220).

[0133] Turning to the watch body 1220, in some examples, the watch body 1220 may have a generally rectangular or circular shape. The watch body 1220 is configured to be worn by a user on their wrist or another body part. More specifically, the watch body 1220 is sized for easy carrying by a user, easy attachment to a part of a user's clothing, and / or easy coupling to a wearable strap 1210 (thus forming a wrist wearable device 1200). As described above, the watch body 1220 may have a shape corresponding to the coupling mechanism 1216 of the wearable strap 1210. In some embodiments, the watch body 1220 includes a single release mechanism 1229 or multiple release mechanisms (e.g., two release mechanisms 1229 positioned on opposite sides of the watch body 1220, such as spring-loaded buttons) to detach the watch body 1220 from the wearable strap 1210. The release mechanism 1229 may include, but is not limited to, buttons, knobs, plugs, handles, levers, fasteners, buckles, dials, latches, or combinations thereof.

[0134] A user can actuate the release mechanism 1229 by pushing, rotating, lifting, pressing, moving, or performing other actions on it. Actuation of the release mechanism 1229 can release (e.g., detach) the watch body 1220 from the coupling mechanism 1216 of the wearable band 1210, allowing the user to use the watch body 1220 independently of the wearable band 1210, and vice versa. For example, detaching the watch body 1220 from the wearable band 1210 allows the user to use the rear camera 1225b to capture images. Although the release mechanism 1229 is shown positioned at a corner of the watch body 1220, it can be positioned anywhere on the watch body 1220 that is convenient for user actuation. Additionally, in some embodiments, the wearable band 1210 may also include a corresponding release mechanism for detaching the watch body 1220 from the coupling mechanism 1216. In some embodiments, the release mechanism 1229 is optional, and as described above, the body 1220 can be detached from the coupling mechanism 1216 (e.g., by twisting, rotating, etc.).

[0135] The watch body 1220 may include one or more peripheral buttons 1223 and 1227 for performing various operations at the watch body 1220. For example, peripheral buttons 1223 and 1227 may be used to turn on or wake up the display 1205 (e.g., to bring the display from sleep to active), unlock the watch body 1220, increase or decrease the volume, increase or decrease the brightness, interact with one or more applications, interact with one or more user interfaces, etc. Additionally or alternatively, in some embodiments, the display 1205 acts as a touchscreen and allows the user to provide one or more inputs for interacting with the watch body 1220.

[0136] In some embodiments, the watch body 1220 includes one or more sensors 1221. The sensors 1221 of the watch body 1220 may be the same as or different from the sensors 1213 of the wearable band 1210. The sensors 1221 of the watch body 1220 may be distributed on the inner and / or outer surfaces of the watch body 1220. In some embodiments, the sensors 1221 are configured to contact the user's skin when the user wears the watch body 1220. For example, the sensors 1221 may be placed on the underside of the watch body 1220, and the coupling mechanism 1216 may be a bracket with an opening that allows the underside of the watch body 1220 to directly contact the user's skin. Alternatively, in some embodiments, the watch body 1220 does not include sensors configured to contact the user's skin (e.g., sensors including those inside and / or outside the watch body 1220, configured to sense data from the watch body 1220 and data from the surrounding environment). In some embodiments, the sensors 1221 are configured to track the position and / or movement of the watch body 1220.

[0137] The watch body 1220 and the wearable band 1210 can share data using wired communication methods (e.g., Universal Asynchronous Receiver / Transmitter (UART), USB transceiver, etc.) and / or wireless communication methods (e.g., Near Field Communication, Bluetooth, etc.). For example, the watch body 1220 and the wearable band 1210 can share data sensed by sensors 1213 and 1221, as well as application and device-specific information (e.g., active and / or available applications, output devices (e.g., display, speaker, etc.), input devices (e.g., touchscreen, microphone, imaging sensor, etc.)).

[0138] In some embodiments, the watch body 1220 may include, but is not limited to, a front-facing camera 1225a and / or a rear-facing camera 1225b, sensors 1221 (e.g., biometric sensors, IMUs, heart rate sensors, oxygen saturation sensors, neuromuscular signal sensors, altimeter sensors, temperature sensors, bioimpedance sensors, pedometer sensors, optical sensors (e.g., imaging sensors 1363), touch sensors, sweat sensors, etc.). In some embodiments, the watch body 1220 may include one or more haptic devices 1376 (e.g., vibratory haptic actuators) configured to provide haptic feedback to the user (e.g., skin sensation and / or kinesthetic perception, etc.). Sensors 1321 and / or haptic devices 1376 may also be configured to operate in conjunction with multiple applications, including but not limited to health monitoring applications, social media applications, gaming applications, and artificial reality applications (e.g., applications associated with artificial reality).

[0139] As described above, the watch body 1220 and the wearable strap 1210, when coupled, can form a wrist wearable device 1200. The watch body 1220 and the wearable strap 1210, when coupled, can function as a single device to perform the functions (operation, detection, communication, etc.) described herein. In some embodiments, each device may be provided with specific instructions for performing one or more operations of the wrist wearable device 1200. For example, if it is determined that the watch body 1220 does not include a neuromuscular signal sensor, the wearable strap 1210 may include alternative instructions for performing associated instructions (e.g., providing sensed neuromuscular signal data to the watch body 1220 via different electronic devices). Operations of the wrist wearable device 1200 may be performed by the watch body 1220 alone or by the watch body in conjunction with the wearable strap 1210 (e.g., via a corresponding processor and / or hardware component), or vice versa. In some embodiments, operations of the wrist wearable device 1200, the watch body 1220, and / or the wearable strap 1210 may be performed in conjunction with one or more processors and / or hardware components.

[0140] For reference below Figure 13 As described in the block diagram, the wearable band 1210 and / or the watch body 1220 may each include independent resources required to perform functions independently. For example, the wearable band 1210 and / or the watch body 1220 may each include a power source (e.g., a battery), memory, data storage device, processor (e.g., a central processing unit (CPU)), communication, light source, and / or input / output devices.

[0141] Figure 13 Block diagrams are shown of a computing system 1330 corresponding to a wearable strap 1210 and a computing system 1360 corresponding to a watch body 1220, according to some embodiments. According to some embodiments, the computing system 1300 of the wrist wearable device 1200 may include a combination of components of the wearable strap computing system 1330 and components of the watch body computing system 1360.

[0142] The watch body 1220 and / or wearable strap 1210 may include one or more components shown in the watch body computing system 1360. In some embodiments, a single integrated circuit may include all or most of the components of the watch body computing system 1360, which are included in a single integrated circuit. Alternatively, in some embodiments, the components of the watch body computing system 1360 may be included in multiple communication-coupled integrated circuits. In some embodiments, the watch body computing system 1360 may be configured (e.g., via a wired or wireless connection) to be coupled to the wearable strap computing system 1330, which may allow the two computing systems to share components, distribute tasks, and / or (individually or as a single device) perform other operations described herein.

[0143] The computing system 1360 may include one or more processors 1379, controllers 1377, peripheral interfaces 1361, power systems 1395, and memory (e.g., memory 1380).

[0144] The power system 1395 may include a charger input 1396, a power-management integrated circuit (PMIC) 1397, and a battery 1398. In some embodiments, the watch body 1220 and the wearable strap 1210 may have their own batteries (e.g., batteries 1398 and 1359) and may share power with each other. The watch body 1220 and the wearable strap 1210 may use various technologies to receive charge. In some embodiments, the watch body 1220 and the wearable strap 1210 may use wired charging components (e.g., a power cord) to receive charge. Alternatively or additionally, the watch body 1220 and / or the wearable strap 1210 may be configured for wireless charging. For example, a portable charging device may be designed to mate with a portion of the watch body 1220 and / or a portion of the wearable strap 1210 and wirelessly deliver available power to the battery 1398 of the watch body 1220 and / or the battery 1359 of the wearable strap 1210. The watch body 1220 and the wearable band 1210 may have independent power systems (e.g., power systems 1395 and 1356, respectively) to enable each to operate independently. The watch body 1220 and the wearable band 1210 may also share power (e.g., one can charge the other) via their respective PMICs (e.g., PMICs 1397 and 1358) and charger inputs (e.g., 1357 and 1396), which can share power via power conductors and ground conductors and / or via a wireless charging antenna.

[0145] In some embodiments, peripheral interface 1361 may include one or more sensors 1321. Sensor 1321 may include one or more coupled sensors 1362 for detecting when the watch body 1220 is coupled to another electronic device (e.g., wearable band 1210). Sensor 1321 may include one or more imaging sensors 1363 (e.g., one or more of the cameras 1325 and / or individual imaging sensors 1363 (e.g., thermal imaging sensors)). In some embodiments, sensor 1321 may include one or more SpO2 sensors 1364. In some embodiments, sensor 1321 may include one or more biopotential signal sensors (e.g., EMG sensors 1365, which may be disposed on the user-facing portion inside the watch body 1220 and / or wearable band 1210). In some embodiments, sensor 1321 may include one or more capacitive sensors 1366. In some embodiments, sensor 1321 includes one or more heart rate sensors 1367. In some embodiments, sensor 1321 may include one or more IMU sensors 1368. In some embodiments, one or more IMU sensors 1368 may be configured to detect movement of the user's hand, or movement of the watch body 1220 in other positions where it is placed or held.

[0146] In some embodiments, one or more of these sensors 1321 may provide an example human-machine interface. For example, a set of neuromuscular sensors, such as EMG sensor 1365, may be arranged circumferentially around wearable band 1210, with the inner surface of EMG sensor 1365 configured to contact the user's skin. Any suitable number of neuromuscular sensors can be used (e.g., 2 to 20 sensors). The number and arrangement of neuromuscular sensors may depend on the specific application using the wearable device. For example, wearable band 1210 may be used to generate control information for controlling augmented reality systems, robots, controlling vehicles, scrolling text, controlling virtual avatars, or any other suitable control task.

[0147] In some embodiments, the neuromuscular sensors may be coupled together using flexible electronics incorporated into a wireless device, and the output of one or more of these sensing elements may optionally be processed using hardware signal processing circuitry (e.g., to perform amplification, filtering, and / or rectification). In other embodiments, at least some of the signal processing on the output of the sensing elements may be performed in software such as processor 1379. Therefore, signal processing on signals sampled by sensors can be performed in hardware, software, or by any suitable combination of hardware and software, as the aspects of the techniques described herein are not limited thereto.

[0148] Neuromuscular signals can be processed in various ways. For example, the output of an EMG sensor 1365 can be provided to an analog front-end, which can be configured to perform analog processing (e.g., amplification, noise reduction, filtering, etc.) on the recorded signal. The processed analog signal can then be provided to an analog-to-digital converter, which can convert the analog signal into a digital signal that can be processed by one or more computer processors. Furthermore, although this example is discussed in the context of an interface with an EMG sensor, the embodiments described herein can also be implemented in wearable interfaces with other types of sensors, including but not limited to mechanomyography (MMG) sensors, sonomyography (SMG) sensors, and electrical impedance tomography (EIT) sensors.

[0149] In some embodiments, the peripheral interface 1361 includes a near-field communication (NFC) component 1369, a global-position system (GPS) component 1370, a long-term evolution (LTE) component 1371, and / or Wi-Fi and / or Bluetooth communication components 1372. In some embodiments, the peripheral interface 1361 includes one or more buttons 1373 (e.g., Figure 12 The peripheral interface 1373 includes peripheral buttons 1223 and 1227, which, when selected by the user, cause an operation to be performed at the body 1220. In some embodiments, the peripheral interface 1361 includes one or more indicators (e.g., light-emitting diodes (LEDs)) to provide the user with visual indicators (e.g., received message, low battery, active microphone and / or camera, etc.).

[0150] The watch body 1220 may include at least one display 1205 for displaying a visual representation of information or data to a user, including user interface elements and / or three-dimensional virtual objects. The display may also include a touchscreen for inputting user input, such as touch gestures and swipe gestures. The watch body 1220 may include at least one speaker 1374 and at least one microphone 1375 for providing audio signals to the user and receiving audio input from the user. The user may provide user input through the microphone 1375 and may also receive audio output from the speaker 1374 as part of a haptic event provided by a haptic controller 1378. The watch body 1220 may include at least one camera 1325, including a front camera 1325a and a rear camera 1325b. The camera 1325 may include an ultra-wide-angle camera, a wide-angle camera, a fisheye camera, a spherical camera, a telephoto camera, a depth-sensing camera, or other types of cameras.

[0151] The watch body computing system 1360 may include one or more haptic controllers 1378 and associated components (e.g., haptic devices 1376) for providing haptic events at the watch body 1220 (e.g., a vibrational sensation or audio output responding to an event at the watch body 1220). The haptic controllers 1378 may communicate with one or more haptic devices 1376 (e.g., electroacoustic devices), which may include speakers in one or more loudspeakers 1374 and / or other audio components and / or electromechanical devices that convert energy into linear motion (e.g., motors, electromagnetic coils, electroactive polymers, piezoelectric actuators, electrostatic actuators, or other haptic output generating components (e.g., components that convert electrical signals into haptic outputs on the device)). The haptic controllers 1378 may provide haptic events that a user of the watch body 1220 can perceive. In some embodiments, the one or more haptic controllers 1378 may receive input signals from one of a plurality of applications 1382.

[0152] In some embodiments, the wearable computing system 1330 and / or the watch body computing system 1360 may include a memory 1380, which may be controlled by one or more storage controllers of the controller 1377. In some embodiments, software components stored in the memory 1380 include one or more applications 1382 configured to perform operations at the watch body 1220. In some embodiments, the one or more applications 1382 may include games, word processors, messaging applications, calling applications, web browsers, social media applications, media streaming applications, financial applications, calendars, clocks, etc. In some embodiments, software components stored in the memory 1380 include one or more communication interface modules 1383 as defined above. In some embodiments, software components stored in the memory 1380 include: one or more graphics modules 1384 for rendering, encoding, and / or decoding audio data and / or visual data; and one or more data management modules 1385 for collecting, organizing, and / or providing access to data 1387 stored in the memory 1380. In some embodiments, one or more applications and / or one or more modules in application 1382 may work together to perform various tasks at table body 1220.

[0153] In some embodiments, the software components stored in the memory 1380 may include one or more operating systems 1381 (e.g., a Linux-based operating system, an Android operating system, etc.). The memory 1380 may also include data 1387. The data 1387 may include data 1388A, sensor data 1389A, media content data 1390, and application data 1391.

[0154] It should be recognized that the table body computing system 1360 is an example of a computing system within the table body 1220, and the table body 1220 may have more or fewer components than those shown in the table body computing system 1360, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the table body computing system 1360 are implemented in hardware, software, firmware, or combinations thereof (including one or more signal processing circuits and / or application-specific integrated circuits).

[0155] Turning to wearable band computing system 1330, one or more components that may be included in wearable band 1210 are shown. Wearable band computing system 1330 may include more or fewer components than those shown in body computing system 1360, may combine two or more components, and / or may have different configurations and / or arrangements of some or all of these components. In some embodiments, all or most of the components of wearable band computing system 1330 are included in a single integrated circuit. Alternatively, in some embodiments, the components of wearable band computing system 1330 are included in multiple communication-coupled integrated circuits. As described above, in some embodiments, wearable band computing system 1330 is configured to be coupled to body computing system 1360 (e.g., via a wired or wireless connection), which allows the two computing systems to share components, assign tasks, and / or (individually or as a single device) perform other operations described herein.

[0156] Similar to the wearable computing system 1360, the wearable computing system 1330 may include: one or more processors 1349; one or more controllers 1347 (including one or more haptic controllers 1348); a peripheral interface 1331, which may include one or more sensors 1313 and other peripheral devices; a power supply (e.g., a power system 1356); and a memory (e.g., a memory 1350), which includes an operating system (e.g., an operating system 1351), data (e.g., data 1354, which includes data 1388B, sensor data 1389B, etc.) and one or more modules (e.g., a communication interface module 1352, a data management module 1353, etc.).

[0157] One or more of these sensors 1313 may be similar to sensor 1321 of the body computing system 1360. For example, sensor 1313 may include one or more coupled sensors 1332, one or more SpO2 sensors 1334, one or more EMG sensors 1335, one or more capacitive sensors 1336, one or more heart rate sensors 1337, and one or more IMU sensors 1338.

[0158] Peripheral interface 1331 may also include other components similar to those included in peripheral interface 1361 of the watch computing system 1360, as described above with reference to peripheral interface 1361. These other components include NFC component 1339, GPS component 1340, LTE component 1341, Wi-Fi and / or Bluetooth communication component 1342, and / or one or more tactile devices 1346. In some embodiments, peripheral interface 1331 includes one or more buttons 1343, a display 1333, a speaker 1344, a microphone 1345, and a camera 1355. In some embodiments, peripheral interface 1331 includes one or more indicators, such as LEDs.

[0159] It should be recognized that the wearable band computing system 1330 is an example of a computing system within the wearable band 1210, and the wearable band 1210 may have more or fewer components than those shown in the wearable band computing system 1330, may combine two or more components, and / or may have different configurations and / or arrangements of these components. The various components shown in the wearable band computing system 1330 may be implemented as one or more of a combination of hardware, software, or firmware (including one or more signal processing circuits and / or application-specific integrated circuits).

[0160] about Figure 12 The wrist wearable device 1200 is an example of a wearable strap 1210 and a watch body 1220 coupled together, and therefore the wrist wearable device 1200 will be understood to include the components shown and described for the wearable strap computing system 1330 and the watch body computing system 1360. In some embodiments, the wrist wearable device 1200 has a split architecture (e.g., a split mechanical architecture, a split electronic architecture, etc.) between the watch body 1220 and the wearable strap 1210. In other words, all the components shown in the wearable strap computing system 1330 and the watch body computing system 1360 may be accommodated or otherwise disposed in the combined wrist wearable device 1200, or accommodated or otherwise disposed in a single component in the watch body 1220, the wearable strap 1210 and / or portions thereof (e.g., the coupling mechanism 1216 of the wearable strap 1210).

[0161] The above technology can be used with any device for sensing neuromuscular signals, but it can also be used with other types of wearable devices for sensing neuromuscular signals, such as body wearables or head wearables that may have neuromuscular sensors closer to the brain or spine.

[0162] In some embodiments, the wrist wearable device 1200 may be used in conjunction with a head wearable device (e.g., AR glasses 1400 and VR system 1510) and / or HIPD, and the wrist wearable device 1200 may also be configured to allow a user to control any aspect of the artificial reality (e.g., by controlling user interface objects in the artificial reality using EMG-based gestures, and / or by allowing a user to interact with a touchscreen on the wrist wearable device to also control aspects of the artificial reality). Having described example wrist wearable devices in this manner, attention now turns to example head wearable devices, such as AR glasses 1400 and VR headset 1510.

[0163] Figures 14 to 16 An example artificial reality system that can be used as a wrist-worn wearable device 1200 or connected to a wrist-worn wearable device 1200 is shown. In some embodiments, such as Figure 14 As shown, the AR system 1400 includes an eye-worn device 1402. In some embodiments, such as Figure 15A and Figure 15B As shown, VR system 1510 includes a head-mounted display (HMD) 1512. In some embodiments, AR system 1400 and VR system 1510 may include one or more simulation components (e.g., components for presenting an interactive artificial reality environment, such as a processor, memory, and / or a presentation device including one or more displays and / or one or more waveguides), some of which reference... Figure 16 A more detailed description is provided. As described herein, a head-mounted wearable device may include components of an eye-worn device 1402 and / or a head-mounted display 1512. Some embodiments of the head-mounted wearable device do not include any display, which includes any of the displays described with reference to AR system 1400 and / or VR system 1510. Although the example artificial reality systems are described herein as AR system 1400 and VR system 1510 respectively, one or both of the example AR systems described herein may be configured to present either a fully immersive virtual reality scene within substantially the entire field of view of the user, or a more subtle augmented reality scene within a portion (less than the entire) of the user's field of view.

[0164] Figure 14 An example visual depiction of an AR system 1400 is shown, which includes an eye-worn device 1402 (which may also be described herein as augmented reality glasses and / or smart glasses). The AR system 1400 may include... Figure 14Additional electronic components, such as wearable accessory devices and / or intermediate processing devices, not shown, are configured to communicate electronically with or otherwise be used in conjunction with the eyewear device 1402. In some embodiments, the wearable accessory devices and / or intermediate processing devices may be configured to communicate via a coupled sensor 1624 ( Figure 16 The electronic communication coupling mechanism is coupled to the eye-wearing device 1402, wherein the coupling sensor 1624 can detect when the electronic device is physically or electronically coupled to the eye-wearing device 1402. In some embodiments, the eye-wearing device 1402 may be configured to be coupled to the housing 1690 ( Figure 16 The housing may include one or more additional coupling mechanisms configured to couple with additional accessory devices. Figure 14 The components shown can be implemented in hardware, software, firmware, or a combination thereof (including one or more signal processing components and / or application-specific integrated circuits (ASICs)).

[0165] The eyewear device 1402 includes a mechanical eyeglass component comprising a frame 1404 configured to hold one or more lenses (e.g., one or both of lenses 1406-1 and 1406-2). Those skilled in the art will recognize that the eyewear device 1402 may include additional mechanical components, such as hinges configured to allow partial folding and unfolding of the frame 1404 of the eyewear device 1402, a bridge configured to span the gap between lenses 1406-1 and 1406-2 and rest on the user's nose, a nose pad configured to rest on the bridge of the nose and provide support for the eyewear device 1402, earpieces configured to rest on the user's ears and provide additional support for the eyewear device 1402, and temple arms configured to extend from the hinges of the eyewear device 1402 to the earpieces, etc. Those skilled in the art will also recognize that some examples of the AR system 1400 may not include the mechanical components described herein. For example, smart contact lenses configured to present artificial reality to users may not include any components of the eyewear device 1402.

[0166] The eye-worn device 1402 includes multiple electronic components, many of which will be discussed below. Figure 16 To provide a more detailed description. Figure 14The diagram illustrates some example electronic components, including acoustic sensors 1425-1, 1425-2, 1425-3, 1425-4, 1425-5, and 1425-6, which may be distributed along a large portion of the frame 1404 of the eye-wear device 1402. The eye-wear device 1402 also includes a left camera 1439A and a right camera 1439B located on different sides of the frame 1404. The eye-wear device 1402 also includes a processor 1448 (or any other suitable type or form of integrated circuit) embedded in a portion of the frame 1404.

[0167] Figure 15A and Figure 15B A VR system 1510, including a head-mounted display (HMD) 1512 (e.g., also referred to herein as an artificial reality head-mounted device, a head-worn device, a VR head-mounted device, etc.), is illustrated according to some embodiments. As mentioned, some artificial reality systems (e.g., AR system 1400) can substantially replace one or more of a user's visual sensory perceptions and / or other sensory perceptions of the real world with virtual experiences (e.g., AR systems 1000 and 1100), rather than blending artificial reality with real reality.

[0168] The HMD 1512 includes a front body 1514 and a frame 1516 (e.g., a strip or strap) shaped to fit the user's head. In some embodiments, the front body 1514 and / or frame 1516 include one or more electronic components (e.g., a display, IMU, tracking transmitter, or detector) for facilitating presentation to and / or interaction with AR and / or VR systems. In some embodiments, such as Figure 15B As shown, the HMD 1512 includes an output audio transducer (e.g., audio transducer 1518). In some embodiments, such as Figure 15B As shown, one or more components (e.g., one or more output audio transducers 1518 and frame 1516) (e.g., frame 1516 and / or part or all of the output audio transducers 1518) can be configured to be attached to and detached from the HMD 1512 (e.g., detachably attached to the HMD 1512). In some embodiments, coupling a detachable component to the HMD 1512 enables the detachable component to enter into electronic communication with the HMD 1512.

[0169] Figure 15A and Figure 15BThe VR system 1510 is also shown to include one or more cameras, such as a left camera 1539A and a right camera 1539B, which may resemble the left and right cameras 1439A and 1439B on the frame 1404 of the eyewear device 1402. In some embodiments, the VR system 1510 includes one or more additional cameras (e.g., cameras 1539C and 1539D) that may be configured to enhance the image data acquired by the left camera 1539A and right camera 1539B by providing more information. For example, camera 1539C may be used to provide color information not identified by cameras 1539A and 1539B. In some embodiments, one or more of the cameras 1539A through 1539D may include an optional infrared (IR) cutoff filter configured to remove IR light received at the respective camera sensor.

[0170] Figure 16 A computing system 1620 and an optional housing 1690 are shown, each of which illustrates multiple components that may be included in the AR system 1400 and / or the VR system 1510. In some embodiments, the optional housing 1690 may include more or fewer components, depending on the actual constraints of the respective AR systems described.

[0171] In some embodiments, the computing system 1620 may include one or more peripheral interfaces 1622A and / or an optional housing 1690 may include one or more peripheral interfaces 1622B. Each of the computing system 1620 and the optional housing 1690 may also include one or more power systems 1642A and 1642B, one or more controllers 1646 (including one or more haptic controllers 1647), one or more processors 1648A and 1648B (as defined above, the one or more processors include any of the examples provided), and memories 1650A and 1650B, all of which may communicate electronically with each other. For example, one or more processors 1648A and 1648B may be configured to execute instructions stored in memories 1650A and 1650B, which may cause one or more controllers in controller 1646 to perform operations at one or more peripheral devices connected to peripheral interfaces 1622A and / or 1622B. In some embodiments, each of the described operations may be powered by power supplied by power systems 1642A and / or 1642B.

[0172] In some embodiments, peripheral interface 1622A may include one or more devices configured to be part of computing system 1620, some of which have already been described above. Figure 12 and Figure 13 The wrist-worn wearable device shown is defined and / or described. For example, peripheral interface 1622A may include one or more sensors 1623A. Some example sensors 1623A include one or more coupled sensors 1624, one or more acoustic sensors 1625, one or more imaging sensors 1626, one or more EMG sensors 1627, one or more capacitive sensors 1628, one or more IMU sensors 1629, and / or any other type of sensor explained above or described with respect to any other embodiments discussed herein.

[0173] In some embodiments, peripheral interfaces 1622A and 1622B may include one or more additional peripheral devices, including one or more NFC devices 1630, one or more GPS devices 1631, one or more LTE devices 1632, one or more Wi-Fi and / or Bluetooth devices 1633, one or more buttons 1634 (e.g., including slide-able or otherwise adjustable buttons), one or more displays 1635A and 1635B, one or more speakers 1636A and 1636B, one or more microphones 1637, one or more cameras 1638A and 1638B (e.g., including a left camera 1639A and / or a right camera 1639B), one or more haptic devices 1640, and / or any other type of peripheral device as defined above or described with respect to any other embodiments discussed herein.

[0174] AR systems can include various types of visual feedback mechanisms (e.g., presentation devices). For example, the display device in AR system 1400 and / or the display device in VR system 1510 can include one or more liquid crystal displays (LCDs), light-emitting diode (LED) displays, organic LED (OLED) displays, and / or any other suitable type of display screen. Artificial reality systems can include (e.g., configured to be seen by both eyes) a single display screen, and / or can provide a separate display screen for each eye, which can allow for additional flexibility for variable focal length adjustment and / or for correcting refractive errors associated with the user's vision. Some embodiments of AR systems also include an optical subsystem with one or more lenses (e.g., conventional concave or convex lenses, Fresnel lenses, or adjustable liquid lenses) through which the user views the display screen.

[0175] For example, corresponding displays 1635A and 1635B may be coupled to each of the lenses 1406-1 and 1406-2 of the AR system 1400. Displays 1635A and 1635B may be coupled to each of the lenses 1406-1 and 1406-2, which may operate together or independently to present an image or a series of images to a user. In some embodiments, the AR system 1400 includes a single display 1635A or 1635B (e.g., a near-eye display) or more than two displays 1635A and 1635B. In some embodiments, a first set of one or more displays 1635A and 1635B may be used to present an augmented reality environment, and a second set of one or more displays 1635A and 1635B may be used to present a virtual reality environment. In some embodiments, one or more waveguides are used in conjunction with presenting artificial reality content to a user of the AR system 1400 (e.g., as a means of delivering light from one or more displays 1635A and 1635B to the user's eyes). In some embodiments, one or more waveguides are wholly or partially integrated into the eye-worn device 1402. In addition to, or in place of, a display screen, some artificial reality systems may include one or more projection systems. For example, the display device in AR system 1400 and / or the display device in VR system 1510 may include (e.g., using waveguides) micro-LED projectors that project light into the display device, such as transparent composite lenses that allow ambient light to pass through. The display device can refract the projected light toward the user's pupil, allowing the user to simultaneously view both artificial reality content and the real world. Artificial reality systems may also be configured with any other suitable type or form of image projection system. In some embodiments, one or more waveguides are additionally or alternatively provided to one or more displays 1635A and 1635B.

[0176] The optional housing 1690 of the computing system 1620 and / or AR system 1400 or VR system 1510 may include some or all of the components of the power systems 1642A and 1642B. The power systems 1642A and 1642B may include one or more charger input terminals 1643, one or more PMICs 1644, and / or one or more batteries 1645A and 1644B.

[0177] Memory 1650A and 1650B may include instructions and data, some or all of which may be stored within memory 1650A and 1650B as a non-transitory computer-readable storage medium. For example, memory 1650A and 1650B may include one or more operating systems 1651, one or more applications 1652, one or more communication interface applications 1653A and 1653B, one or more graphics applications 1654A and 1654B, one or more AR processing applications 1655A and 1655B, and / or any other type of data as defined above or described with respect to any other embodiments discussed herein.

[0178] Memory 1650A and 1650B also include data 1660A and 1660B, which can be used in conjunction with one or more of the applications discussed above. Data 1660A and 1660B may include data 1661, sensor data 1662A and 1662B, media content data 1663A, AR application data 1664A and 1664B, and / or any other type of data as defined above or described with respect to any other embodiments discussed herein.

[0179] In some embodiments, the controller 1646 of the eye-worn device 1402 can process information generated by sensors 1623A and / or 1623B on the eye-worn device 1402, and / or another electronic device within the AR system 1400. For example, the controller 1646 can process information from acoustic sensors 1425-1 and 1425-2. For each detected sound, the controller 1646 can perform direction-of-arrival (DOA) estimation to estimate the direction in which the detected sound arrives at the eye-worn device 1402 of the AR system 1400. When one or more of the plurality of acoustic sensors 1625 (e.g., acoustic sensors 1425-1, 1425-2) detect sound, the controller 1646 can use information (e.g., in...) to... Figure 16 The audio dataset is filled with sensor data (represented as sensor data 1662A and 1662B).

[0180] In some embodiments, physical electronic connectors can transmit information between the eye-worn device 1402 and another electronic device and / or between one or more processors 1448, 1648A, 1648B and controller 1646 in the AR system 1400 or VR system 1510. This information can be in the form of optical data, electronic data, wireless data, or any other transmissible data format. Moving the processing of information generated by the eye-worn device 1402 to an intermediate processing device can reduce weight and heat in the eye-worn device, making it more comfortable and safer for the user. In some embodiments, optional wearable accessory devices (e.g., electronic neckbands) are coupled to the eye-worn device 1402 via one or more connectors. These connectors can be wired or wireless connectors and can include electronic components and / or non-electronic (e.g., structural) components. In some embodiments, the eye-worn device 1402 and the wearable accessory devices can operate independently without any wired or wireless connection between them.

[0181] In some cases, external devices such as intermediate processing devices (e.g., HIPD 806, 906, 1006) paired with the eyewear device 1402 (e.g., as part of the AR system 1400) enable the eyewear device 1402 to achieve similar shape elements to a pair of glasses while still providing sufficient battery power and computing power for extended capabilities. Some or all of the battery power, computing resources, and / or additional features of the AR system 1400 may be provided by the paired device or shared between the paired device and the eyewear device 1402, thereby reducing the overall weight, heat distribution, and shape elements of the eyewear device 1402 while allowing the eyewear device 1402 to retain its required functionality. For example, wearable accessory devices may allow components originally included in the eyewear device 1402 to be included in the wearable accessory device and / or intermediate processing device, thereby transferring the weight load from the user's head and neck to one or more other parts of the user's body. In some embodiments, the intermediate processing device has a large surface area through which heat is diffused and dispersed to the surrounding environment. Therefore, the intermediate processing device allows for greater battery capacity and computing power compared to what might have been possible on a standalone eye-worn device 1402. Because the weight carried in the wearable accessory is less invasive to the user than the weight carried in the eye-worn device 1402, users can tolerate wearing a lighter eye-worn device and carrying or wearing the paired device for longer periods compared to tolerating wearing a heavier eye-worn device alone. This allows for a more complete integration of the artificial reality environment into the user's daily activities.

[0182] AR systems can include various types of computer vision components and subsystems. For example, AR system 1400 and / or VR system 1510 can include one or more optical sensors, such as two-dimensional (2D) or three-dimensional (3D) cameras, time-of-flight depth sensors, structured light emitters and detectors, single-beam or scanning laser rangefinders, 3D LiDAR sensors, and / or any other suitable type or form of optical sensor. AR systems can process data from one or more of these sensors to identify the user's location and / or aspects of the user's real-world physical environment, including the locations of real-world objects within the real-world physical environment. In some embodiments, among various other functions, the methods described herein are used to map the real world, provide the user with context about the real-world environment, and / or generate digital twins (e.g., interactive virtual objects). For example, Figure 15A and Figure 15B A VR system 1510 with cameras 1539A to 1539D is shown. These cameras can be used to provide depth information for creating voxel fields and two-dimensional meshes to provide the user with object information to avoid collisions.

[0183] In some embodiments, AR system 1400 and / or VR system 1510 may include a haptic feedback system, which may be integrated into headwear, gloves, bodysuits, handheld controllers, environmental devices (e.g., chairs or footrests) and / or any other type of device or system, such as wearable devices discussed herein. The haptic feedback system can provide various types of skin feedback (including vibration, force, traction, shear, texture, and / or temperature). The haptic feedback system can also provide various types of kinematic feedback, such as motion and compliance. Haptic feedback can be implemented using motors, piezoelectric actuators, fluid systems, and / or various other types of feedback mechanisms. The haptic feedback system can be implemented independently of other artificial reality devices, within other artificial reality devices, and / or in conjunction with other artificial reality devices.

[0184] In some embodiments of artificial reality systems such as AR system 1400 and / or VR system 1510, ambient light (e.g., a real-time feed of the surrounding environment that the user will typically see) can pass through the display elements of the respective head-mounted wearable devices that present aspects of the AR system. In some embodiments, ambient light can pass through a portion (less than all) of the AR environment presented within the user's field of view (e.g., a portion of the AR environment is located in the same location as physical objects in the user's real-world environment, which is within a designated boundary (e.g., a guardian boundary) configured for use by the user when interacting with the AR environment). For example, visual user interface elements (e.g., notification user interface elements) can be presented on the head-mounted wearable device, and a certain amount of ambient light (e.g., 15% to 50% of the ambient light) can pass through the user interface elements, allowing the user to distinguish at least a portion of the physical environment on which the user interface elements are displayed.

[0185] The order of process parameters and steps described and / or illustrated herein is given by way of example only and may be changed as needed. For example, although the steps shown and / or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the shown or discussed order. The various exemplary methods described and / or illustrated herein may also omit one or more of the steps described or illustrated herein, or include additional steps in addition to the steps disclosed herein.

[0186] The foregoing description has been provided to enable others skilled in the art to best utilize various aspects of the exemplary embodiments disclosed herein. This exemplary description is not intended to be exhaustive or limiting to any precise form disclosed. Many modifications and variations are possible without departing from the scope of this disclosure. The embodiments disclosed herein should be considered illustrative rather than restrictive in all respects. Reference should be made to any appended claims and their equivalents in determining the scope of this disclosure.

[0187] Unless otherwise stated, the terms “connected to” and “coupled to” (and their derivatives) as used in the specification and / or claims shall be interpreted as allowing both direct and indirect (i.e., via other elements or components) connections. Additionally, the terms “a” or “an” as used in the specification and / or claims shall be interpreted as meaning “at least one of…”. Finally, for ease of use, the terms “comprising” and “having” (and their derivatives) as used in the specification and / or claims are interchangeable with the word “including” and have the same meaning.

[0188] It will be understood that when an element (e.g., a layer or region) is said to be formed on, deposited on, or positioned “on” or “above” another element, that element may be directly on at least a portion of that other element, or one or more intermediate elements may be present. Conversely, when an element is said to be “directly on” or “directly above” another element, that element may be on at least a portion of that other element without any intermediate elements.

[0189] As used herein, in some embodiments, the term "about" as used with respect to a particular numerical value or range of values ​​may refer to and include the value as well as all values ​​within 10% of the value. Thus, for example, in some embodiments, the numerical value "50" referred to as "about 50" may include values ​​equal to 50 ± 5, i.e., values ​​in the range of 45 to 55.

[0190] As used herein, the term "substantially" in relation to a given parameter, performance, or condition may refer to and include the extent that a person skilled in the art would understand, i.e., that the given parameter, performance, or condition is satisfied with a small degree of variation, such as within acceptable manufacturing tolerances. As an example, depending on the specific parameter, performance, or condition being substantially satisfied, it may be satisfied with at least about 90%, at least about 95%, or even at least about 99%.

[0191] While the transitional phrase "comprising" may be used to disclose various features, elements, or steps of a particular embodiment, it should be understood that alternative embodiments (including those described using the transitional phrases "consisting of" or "substantially consisting of") are implicit. Thus, for example, implicit alternative embodiments of a dielectric waveguide comprising or including lithium niobate include embodiments where the dielectric waveguide is substantially composed of lithium niobate and embodiments where the dielectric waveguide is composed of lithium niobate.

Claims

1. A display system, the display system comprising: A waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end via total internal reflection; A coupling mirror, the coupling mirror being configured to guide the image light into the waveguide body; A rotating mirror, the rotating mirror being configured to rotate the image light within the waveguide body; A refractive mirror configured to propagate the image light within the waveguide body; as well as A coupling mirror is configured to guide the image light out of the waveguide body.

2. The display system according to claim 1, wherein, The orientation of the input field of view of the display system and the orientation of the output field of view of the display system are the same.

3. The display system according to claim 1 or 2, wherein, The orientation of the coupling mirror is the same as that of the coupling mirror.

4. The display system according to any of the preceding claims, wherein, The tilt angle of the coupling mirror is equal to the tilt angle of the coupling mirror.

5. The display system according to any of the preceding claims, wherein, The clock angle of the coupled-in mirror is equal to the clock angle of the coupled-out mirror.

6. The display system according to any of the preceding claims, wherein, The orientation of the rotating mirror is the same as that of the refractive mirror.

7. The display system according to any of the preceding claims, wherein, The tilt angle of the rotating mirror is equal to the tilt angle of the refracting mirror.

8. The display system according to any of the preceding claims, wherein, The clock angle of the rotating mirror is equal to the clock angle of the refractive mirror.

9. The display system according to any of the preceding claims, wherein, The rotating mirror has a reflectivity of 90% to 100%.

10. The display system according to any of the preceding claims, wherein any of the following is present: a) wherein the rotating mirror comprises a single reflective element; or b) Among them, The rotating mirror includes multiple mirrors arranged in the mirror area.

11. The display system according to any of the preceding claims, the display system further comprising a projector for providing the image light to the waveguide body.

12. The display system according to claim 11, wherein one or more of the following are present: a) wherein the rotating reflector is configured to align the field of view of the projector with the field of view of the user; or b) The display system further includes an alignment prism located between the projector and the waveguide body, wherein, The alignment prism is configured to align the projector with at least one of the tilt angle and the clock angle relative to the waveguide body.

13. A display system, the display system comprising: Projector, the projector being configured to provide image light; A waveguide configured to guide the image light from its input end to its output end; A coupling mirror is located near the input end of the waveguide and is configured to guide the image light into the waveguide; A rotating mirror configured to rotate the image light within the waveguide; A refractive mirror configured to propagate the image light within the waveguide; as well as A coupling mirror is located near the output end of the waveguide and is configured to guide the image light out of the waveguide.

14. The display system according to claim 13, wherein any of the following is present: a) Wherein, the orientation of the input field of view of the display system and the orientation of the output field of view of the display system are equivalent; or b) Among them, The rotating mirror includes a single reflective element; or c) Wherein, the rotating reflector comprises a plurality of reflectors arranged in the reflector region; or d) The display system further includes an alignment prism located between the projector and the waveguide, wherein the alignment prism is configured to align the projector with at least one of a tilt angle and a clock angle relative to the waveguide.

15. A display system, the display system comprising: A waveguide body extending from an input end to an output end and configured to guide image light from the input end to the output end via total internal reflection; A coupling mirror, the coupling mirror being configured to guide the image light into the waveguide body; A rotating mirror, the rotating mirror being configured to rotate the image light within the waveguide body; A refractive mirror configured to propagate the image light within the waveguide body; as well as A coupling reflector is configured to guide the image light out of the waveguide body. Wherein, the orientation of the coupling mirror and the orientation of the coupling mirror are the same; and The orientation of the rotating mirror is the same as that of the refractive mirror.

Images