Rotary XR glasses display

Abstract

An augmented reality (XR) display system includes a light emitting diode (LED) display controller and a light emitting diode (LED) near-eye display element operatively coupled to an LED display driver. The LED near-eye display element includes one or more motors and an array of LEDs operably connected to the one or more motors. During operation, the LED display driver receives video data, which includes a rendered virtual object of the XR experience, and generates an LED array control signal based on the video data, the LED array control signal causing one or more LEDs of the LED array to be sequentially energized. The LED display driver also generates a synchronous motor control signal and simultaneously transmits the LED array control signal to the LED array and the synchronous motor control signal to the one or more motors, causing the LED near-eye display element to display the rendered virtual object.

Description

[0001] Priority Claim

[0002] This application claims the benefit of priority of Greek Patent Application No. 20220100979, filed on November 30, 2022, and U.S. Patent Application No. 18 / 128,905, filed on March 30, 2023, the entire disclosures of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to user interfaces and, more particularly, to augmented reality and virtual reality using user interfaces. Background Art

[0004] A head-mounted device can be implemented using a transparent or translucent display through which a user of the head-mounted device can view the surrounding environment. Such a device enables the user to view the surrounding environment through the transparent or translucent display and also to see objects (e.g., rendered virtual objects such as 2D or 3D graphical models, images, videos, text, etc.) that are generated for display and appear as part of and / or overlaid on the surrounding environment. This is generally referred to as "augmented reality" or "AR". A head-mounted device can also completely occlude the user's field of view and display a virtual environment through which the user can move or be moved. This is generally referred to as "virtual reality" or "VR". In a hybrid form, a view of the surrounding environment is captured using a camera device and then displayed to the user together with augmentation on a display that occludes the user's eyes. As used herein, unless the context otherwise indicates, the term extended reality (XR) refers to augmented reality, virtual reality, and any hybrid of these technologies.

[0005] In order to interact with XR applications provided by an XR system or XR device, it is desirable to have a user interface. Brief Description of the Drawings

[0006] To facilitate identification of discussion of any particular element or action, one or more of the most significant digits in the reference numerals refer to the figure number in which the element is first introduced.

[0007] Figure 1A is a perspective view of a head-mounted device according to some examples.

[0008] Figure 1B Shows a head-mounted device according to some examples of Figure 1A additional views of.

[0009] Figure 2A is an illustration of an LED array according to some examples.

[0010] Figure 2B And Figure 2C is a top view of an optical element holder for holding an LED near-eye display element according to some examples.

[0011] Figure 3 is a diagram of an optical element holder and a corresponding LED near-eye display element according to some examples.

[0012] Figure 4A shows a front view of an alternative LED near-eye display element according to some examples, and Figure 4B shows a top view of the alternative LED near-eye display element.

[0013] Figure 5 is a diagram of another optical element holder and a corresponding LED near-eye display element according to some examples.

[0014] Figure 6 is an illustration of another optical element holder and a corresponding LED near-eye display element according to some examples.

[0015] Figure 7A is an architecture diagram of an XR display system according to some examples, and Figure 7B shows an example XR display system method.

[0016] Figure 8 is a graphical representation of a machine within which instructions can be executed to cause the machine to perform any one or more of the methods discussed herein.

[0017] Figure 9 is a block diagram showing a software architecture within which the present disclosure may be implemented according to some examples.

[0018] Figure 10 is a block diagram showing a networked system including details of a head-mounted XR system according to some examples.

[0019] Figure 11 is a graphical representation of a networked environment in which examples of the present disclosure may be deployed according to some examples. Detailed Description

[0020] Some perspective XR display systems use optically based waveguides. These waveguides may have a small field of view (FOV), may be expensive, and may consume a large amount of energy. Accordingly, there is a need for an XR display system that has a larger FOV, lower production costs, and consumes less energy.

[0021] In one aspect, an XR display system includes a light-emitting diode (LED) display controller and a light-emitting diode (LED) near-eye display element operatively coupled to an LED display driver. The LED near-eye display element includes one or more motors and an LED array operatively connected to the one or more motors. The LED array can be less expensive compared to waveguides and can be manufactured in any size to create a near-eye display with a large FOV. Additionally, the power requirements of the LED array and motors can be lower than those of a similar waveguide system.

[0022] During operation, video data of an XR experience is used to generate an LED array control signal that causes one or more LEDs of the LED array to be energized in sequence. Additionally, a synchronous motor control signal is generated based on the video data, and the synchronous motor control signal causes the one or more motors to be powered synchronously with the energization of the one or more LEDs. The LED array control signal is transmitted to the LED array, and simultaneously, the synchronous motor control signal is transmitted to the one or more motors, causing the LED near-eye display element to display the XR experience to the user.

[0023] In some examples, the one or more motors cause the LED array to sweep across a circular sweep area, a sector of the circular sweep area, or a rectangular sweep area.

[0024] In some examples, the LED array of the LED near-eye display element is operatively coupled to the one or more motors via a magnetic ring gear and one or more magnetic pinion gears.

[0025] In some examples, the LED array is powered by inductive coupling.

[0026] In some examples, the LED array is controlled via a wireless communication protocol.

[0027] Other technical features will be apparent to those skilled in the art from the following drawings, description, and claims.

[0028] Figure 1Ais a perspective view of a head-mounted XR system 100 according to some examples. The head-mounted XR system 100 may include a frame 102 made of any suitable material such as plastic or metal, and the any suitable material includes any suitable shape memory alloy. In one or more examples, the frame 102 includes a first optical element holder or first optical element holder 104 (e.g., a display or lens holder) and a second optical element holder or second optical element holder 106 connected by a bridge portion 112. A left light-emitting diode (LED) near-eye display element or first light-emitting diode (LED) near-eye display element 108 and a right LED near-eye display element or second LED near-eye display element 110 may be disposed within the corresponding first optical element holder 104 and second optical element holder 106, respectively.

[0029] The frame 102 also includes a left arm member or left temple member 122 and a right arm member or right temple member 124. In some examples, the frame 102 may be formed from a single piece of material to have a unified or integral construction.

[0030] The head-mounted XR system 100 may include a computing system such as a computer 120, which may be of any suitable type to be carried by the frame 102, and in one or more examples, it may be of a suitable size and shape to be partially disposed within one of the left temple member 122 or right temple member 124. The computer 120 may include multiple processors, memories, and various communication components that share a common power supply. As described below, various components of the computer 120 may include low-power circuitry, high-speed circuitry, and a display processor. Various other examples may include these elements configured differently or integrated together in different ways. Additional details of aspects of the computer 120 may be implemented as shown by the data processor 1002 discussed below.

[0031] The computer 120 also includes a battery 118 or other suitable portable power supply. In some examples, the battery 118 is disposed within the left temple member 122 and electrically coupled to the computer 120 disposed within the right temple member 124. The head-mounted XR system 100 may include a connector or port (not shown) suitable for charging the battery 118, a wireless receiver, a transmitter, or a transceiver (not shown), or a combination of such devices.

[0032] The head-mounted XR system 100 includes a first imaging device or left imaging device 114 and a second imaging device or right imaging device 116. Although two imaging devices are depicted, other examples contemplate the use of a single or additional (i.e., more than two) imaging devices. In one or more examples, in addition to the left imaging device 114 and the right imaging device 116, the head-mounted XR system 100 further includes any number of input sensors or other input / output devices. Such sensors or input / output devices may additionally include biometric sensors, position sensors, motion sensors, and the like.

[0033] In some examples, the left imaging device 114 and the right imaging device 116 provide video frame data for the head-mounted XR system 100 to extract 3D information from a real-world scene environment scene.

[0034] The head-mounted XR system 100 may also include a touchpad 126 that is mounted to or integrated with one or both of the left temple piece 122 and the right temple piece 124. The touchpad 126 is typically arranged vertically and, in some examples, is approximately parallel to the user's temple. As used herein, being typically vertically aligned means that the touchpad is more vertical compared to the horizontal, although potentially more vertical than said vertical. Additional user input may be provided by one or more buttons 128, which in the example shown are disposed on the outer upper edges of the first optical element holder 104 and the second optical element holder 106. The one or more touchpads 126 and buttons 128 provide means by which the head-mounted XR system 100 can receive input from a user of the head-mounted XR system 100.

[0035] Figure 1B The head-mounted XR system 100 is shown from the perspective of the user. For clarity, several elements shown in Figure 1A are omitted. As Figure 1A described, Figure 1B the head-mounted XR system 100 shown includes a first optical element holder 104 and a second optical element holder 106 that respectively hold the first LED near-eye display element 108 and the second LED near-eye display element 110 fixed therein.

[0036] The first LED near-eye display element 108 includes a rotating rim 136 and an LED array 130 that is attached to the rotating rim 136 at a first distal portion of the LED array 130 and a second distal portion of the LED array 130. The LED array 130 spans an interior portion of the rotating rim 136. The rotating rim 136 is enclosed by the first LED near-eye display element 108 and rotates (as indicated by arrow 138a and arrow 138b) directly or indirectly within the first LED near-eye display element 108 by one or more motors (not shown). In some examples, the rotating rim 136 rotates counterclockwise by one or more motors. In some examples, the rotating rim 136 rotates clockwise by one or more motors.

[0037] The second optical element holder 106 holds a second LED near-eye display element 110 that includes a rotating rim 132 and an LED array 134 that is attached to the rotating rim 132 at a first distal portion of the LED array 134 and a second distal portion of the LED array 134. The LED array 134 spans an interior portion of the rotating rim 132. The rotating rim 132 is enclosed by the second LED near-eye display element 110 and rotates (as indicated by arrow 140a and arrow 140b) within the second LED near-eye display element 110 directly or indirectly by one or more motors (not shown). In some examples, the rotating rim 132 rotates counterclockwise by one or more motors. In some examples, the LED array 134 rotates clockwise by one or more motors.

[0038] In some examples, components of the LED near-eye display element include a transparent substrate on which the LED array is mounted. In some examples, the transparent substrate is circular and an outer rim portion of the transparent substrate includes the rotating rim. In some examples, the LED array is formed in the transparent substrate.

[0039] The combination of an LED display driver (not shown), the first LED near-eye display element 108, and the second LED near-eye display element 110 provides an XR display system that is a component of the head-mounted XR system 100. The head-mounted XR system 100 uses the XR display system to generate an overlay of a view of the user's real-world scene environment that includes a display of a user interface for the user of the head-mounted XR system 100.

[0040] In use, a user of the head-mounted XR system 100 can be presented with information, content, and various user interfaces on the LED near-eye display element, as described in more detail herein. The user can then interact with the head-mounted XR system 100 using the touchpad 126 and / or buttons 128, voice input or touch input on an associated device (e.g., the mobile computing system 1026 shown in Figure 10 ), and / or hand movements, positions, and orientations detected by the head-mounted XR system 100.

[0041] In some examples, the head-mounted XR system 100 includes a stand-alone XR system that provides an XR experience to a user of the head-mounted XR system 100. In some examples, the head-mounted XR system 100 is a component of an XR system that includes one or more other devices that provide additional computing resources and / or additional user input and output resources. The other devices can include smart phones, general-purpose computers, and the like.

[0042] Figure 2A is a diagram of an LED array according to some examples. The LED array 202 can be used as the Figure 1B LED array 130 or LED array 134. The LED array 202 includes one or more LEDs arranged in a linear pattern along the long axis of the LED array 202. In some examples, the one or more LEDs are grouped into separate LED groups of red, green, and blue. In some examples, the one or more LEDs are red, green, blue (RGB) LEDs. The one or more LEDs are controlled by an LED array control signal receiver 208 that includes LED array control logic operable to control the sequence of energizing the one or more LEDs of the LED array 202. In some examples, the LED array 202 and the LED array control signal receiver 208 and the LED array control logic are powered by an LED array power receiver circuit 210 inductively coupled to the power supply of the XR system.

[0043] Figure 2B and Figure 2C are top views of an optical element holder that holds an LED near-eye display element according to some examples. As shown in Figure 2B , the planar optical element holder 204 can hold the planar LED near-eye display element 212. As shown in Figure 2C , the curved optical element holder 206 can hold the curved LED near-eye display element 214. In some examples, the LED near-eye display element can include one or more vision correction components for correcting the vision of the user.

[0044] Figure 3 is a diagram of an optical element holder and a corresponding LED near-eye display element according to some examples.

[0045] The optical element holder 306 holds the LED near-eye display element 308, which has a rotating rim including a magnetic ring gear 304 and an LED array 310. The LED array 310 is attached to the magnetic ring gear 304 at a first distal portion of the LED array 310 and a second distal portion of the LED array 310. The LED array 310 spans an inner portion of the magnetic ring gear 304. The magnetic ring gear 304 is surrounded by the LED near-eye display element 308 and is rotated or revolved (as indicated by arrows 312a and 312b) within the LED near-eye display element 308 by one or more magnetic pinions such as magnetic pinions 302a, magnetic pinions 302b, magnetic pinions 302c, and magnetic pinions 302d. The magnetic pinions are driven by one or more motors (not shown). In some examples, the magnetic ring gear 304 rotates in a counterclockwise direction by one or more magnetic pinions. In some examples, the magnetic ring gear 304 rotates in a clockwise direction by one or more magnetic pinions.

[0046] In certain examples, a non-magnetic ring gear is used. In some examples, non-magnetic pinions are used.

[0047] In some examples, the optical element holder 306 holds an LED near-eye display element having a rotating rim and an LED array attached to the rotating rim at a first distal portion of the LED array and a second distal portion of the LED array. The LED array spans an inner portion of the rotating rim. The rotating rim is surrounded by the LED near-eye display element and is rotated within the LED near-eye display element by one or more powered rollers. The powered rollers are driven by one or more motors.

[0048] Figure 4A A front view of an alternative LED near-eye display element according to some examples is shown, and Figure 4B A top view of the alternative LED near-eye display element is shown. The alternative LED near-eye display element 404 spans the face of a user covering the user's left and right eyes with a single LED near-eye display element having one or more LED arrays.

[0049] The optical element holder 402 holds the LED near-eye display element 404, which includes a rotating rim 406 and an LED array 408. The LED array is attached to the rotating rim 406 at a first distal portion and a second distal portion of the LED array 408. The LED array 408 spans an inner portion of the rotating rim 406. The rotating rim 406 is surrounded by the LED near-eye display element 404 and rotates or turns within the LED near-eye display element 404. In some examples, the rotating rim 406 rotates in a counterclockwise direction by one or more motors. In some examples, the rotating rim 406 rotates in a clockwise direction by one or more motors.

[0050] Rotation of the LED array 408 causes the LED array 408 to sweep through a circular sweep region. The circular sweep region includes a left viewing sector 416 visible to the user's left eye and a right viewing sector 418 visible to the user's right eye.

[0051] In some examples, the components of the LED near-eye display element include a transparent substrate, and one or more LED arrays and the rotating rim are mounted on the transparent substrate. In some examples, the transparent substrate is circular, and an outer rim portion of the transparent substrate includes the rotating rim. In some examples, one or more LED arrays are formed in the transparent substrate.

[0052] In some examples, the optical element holder 402 includes a central mounting post 412 that extends from a rim portion of the optical element holder 402 to a central portion of the optical element holder 402. One or more motors are mounted on the central mounting post 412 and are used to rotate the LED array 408.

[0053] As Figure 4B shown, the curved optical element holder 410 can hold the curved LED near-eye display element 414. In some examples, the LED near-eye display element can include one or more vision correction components for correcting the user's vision.

[0054] Figure 5Diagram of another optical element holder and corresponding LED near-eye display element according to some examples. The optical element holder 502 holds the LED near-eye display element 504. The LED near-eye display element 504 includes an LED array 506, which is attached to one or more traction belts such as traction belts 510a and 510b. The LED array 506 is attached to the first traction belt such as traction belt 510a at the first distal portion, and is attached to the second traction belt such as traction belt 510b at the second distal portion. The traction belts are operatively attached to one or more powered rollers, such as powered roller 512a or powered roller 512b, and optionally attached to one or more unpowered rollers, such as unpowered roller 514a or unpowered roller 514b. The traction belts are operable to transport the LED array 506 back and forth between positions such as LED array position 508a and LED array position 508b, thereby forming a rectangular sweep area 516, in which an image can be formed by selectively energizing one or more LEDs of the LED array 506.

[0055] In some examples, the rotation directions of the powered rollers 512a and 512b are switched to change the movement directions of the traction belts 510a and 510b to position the LED array 506 in a specified position.

[0056] In some examples, the LED array 506 is fixed to the traction belts 510a and 510b by rotatable connectors. The powered rollers 512a and 512b are driven in a single rotation direction. The rotatable connectors allow the traction belts 510a and 510b to travel around the rollers 512a, 512b, 514a, 514b while attaching to the distal portions of the LED array 506.

[0057] In some examples, two or more LED arrays are attached to one or more traction belts.

[0058] Figure 6 Diagram of another optical element holder and corresponding LED near-eye display element according to some examples. The optical element holder 608 holds the LED near-eye display element 606. The LED near-eye display element 606 includes the LED near-eye display element 606, with the first distal portion of the LED array 604 attached to the motor 602, while its second distal portion remains free. The motor 602 is operable to oscillate the LED array 604 back and forth as indicated by the arc 610, causing the LED array to sweep across a sector 612 of a circular sweep area.

[0059] Figure 7A Architecture diagram of an XR display system according to some examples, and Figure 7BAn example XR display system method is shown. Although the example XR display system method 700 depicts a particular order of operations, the order can be changed without departing from the scope of the present disclosure. For example, some of the depicted operations can be performed in parallel or in a different order that does not substantially affect the functionality of the XR display system method 700. In other examples, different components of an example device or system implementing the XR display system method 700 can perform functions substantially simultaneously or in a particular order.

[0060] In operation 702, the LED display driver 736 of the XR display system 740 receives video data 724 of an XR experience generated by an XR application 714 of the XR system. The video data 724 includes the rendering of one or more virtual objects of the XR experience provided by the XR application 714. For example, the XR application 714 generates virtual object data 726 of the XR experience provided to a user of the XR system. The virtual object data 726 includes data of one or more virtual objects, such as, but not limited to, 3D geometric data of the size, shape, and position of one or more virtual objects that are virtually located in a real-world scene of the XR experience. The virtual object data 726 also includes graphical information of one or more virtual objects, such as color, shading, and texture. The XR application 714 transmits the virtual object data 726 to the graphics engine 712 of the XR system.

[0061] The graphics engine 712 receives the virtual object data 726 and generates the video data 724 based on the virtual object data 726. The video data 724 includes 3D renderings of one or more virtual objects of the XR experience as they would appear in a real-world scene according to the perspective of a user of the XR system. The graphics engine 712 transmits the video data 724 to the display element controller 710 of the LED display driver 736 of the XR display system 740.

[0062] In operation 704, the LED display driver 736 generates an LED array control signal 722 based on the video data 724. The LED array control signal 722 causes one or more LEDs in one or more LED arrays 732 of one or more LED near-eye display elements 738 to be energized in sequence. For example, the display element controller 710 of the LED display driver 736 generates an LED sequence instruction 728 based on the video data 724. The LED sequence instruction 728 includes information on how to sequentially energize one or more LEDs in one or more LED arrays 732 as the one or more LED arrays 732 are moved in front of a user's eyes by one or more motors 720 of one or more LED near-eye display elements 738. When the movement of one or more LEDs is correctly synchronized with the sequential energization of one or more LEDs, an image visible to the user is formed.

[0063] In some examples, a motor of the LED near-eye display element rotates an LED array of the LED near-eye display element about a central axis of the LED array and at a specified rotational speed. LEDs of the LED array have a known distance from a center of rotation of the LED array. When the LED array rotates or turns at the specified rotational speed, a circular sweep area swept by the LED array during the rotation can be represented as having respective pixels corresponding to respective LEDs of the LED array. A position of a pixel within the circular sweep area can be represented in a polar coordinate system as a rotational angle of the LED array in the polar coordinate system and a distance of the pixel from the center of rotation of the LED array. Video data generated by a graphics engine includes video frame data that includes pixel data of respective pixels including an X coordinate and a Y coordinate in a Cartesian coordinate system. A display element controller 710 maps pixels having Cartesian coordinates in the video frame data of the video data 724 to pixels having polar coordinates in the circular sweep area of the LED near-eye display element. The display element controller includes polar coordinates of the mapped pixels in an LED sequence instruction 728 that is transmitted to other components of the XR display system.

[0064] In some examples, the motor rotates the LED array through an arc, forming a sweep area that is a sector of the circular sweep area.

[0065] In some examples, a linear LED array moves back and forth in a linear motion having a direction of motion orthogonal to a linear axis of the LED array. Accordingly, the LED array sweeps a rectangular sweep area. The display element controller 710 maps pixels having Cartesian coordinates in the video frame data of the video data 724 to Cartesian coordinates of the rectangular sweep area of the linear LED array.

[0066] The display element controller 710 transmits the LED sequence instruction 728 to one or more LED array drivers 716. The one or more LED array drivers 716 receive the LED sequence instruction 728 and generate an LED array control signal 722 based on the LED sequence instruction 728.

[0067] In operation 706, the LED display driver 736 generates a synchronous motor control signal 734 based on video data. The synchronous motor control signal 734 powers one or more motors 720 of one or more LED near-eye display elements 738 in synchronization with one or more LED arrays 732. For example, the display element controller 710 generates a synchronous motor control instruction 730 that is synchronized with the LED sequence instruction 728 based on the video data 724. For example, in the case where one or more motors of the LED near-eye display element rotate the LED array 360 degrees and the video data 724 includes video frame data at 30 frames per second (fps), the LED sequence instruction 728 includes 30 frame data to be displayed within one second, where the frames are displayed during a complete rotation of the LED array. Thus, the synchronous motor control instruction 730 includes instructions that direct one or more motor drivers 718 to operate one or more corresponding motors to rotate the LED array at 30 rotations per second or 1800 revolutions per minute (RPM). The display element controller 710 transmits the synchronous motor control instruction 730 to one or more motor drivers 718. One or more motor drivers 718 receive the synchronous motor control instruction 730 and generate the synchronous motor control signal 734 based on the synchronous motor control instruction 730.

[0068] In operation 708, the LED display driver 736 simultaneously transmits an LED array control signal 722 to one or more LED arrays of the LED array 732 and transmits the synchronous motor control signal to one or more motors of the motors 720, causing one or more LED near-eye display elements 738 to display a rendered virtual object to a user of the XR system. For example, one or more LED array drivers 716 of the LED display driver 736 transmit the LED array control signal 722 to one or more LED array control signal receivers 744 connected to one or more corresponding LED arrays 732. One or more LED array control signal receivers 744 include LED array control logic that controls the energization of one or more LEDs of one or more corresponding LED arrays 732 based on the LED array control signal 722. In addition, one or more motor drivers 718 transmit the synchronous motor control signal 734 to one or more motors 720 of one or more LED near-eye display elements 738. When one or more LED array control signal receivers 744 energize the corresponding LED arrays 732 connected to one or more corresponding motors 720 to display the rendered virtual object of the XR experience to a user of the XR system, one or more motors 720 move one or more LED arrays 732.

[0069] In some examples, an XR display system includes a left (first) LED near-eye display element and a right (second) LED near-eye display element. A graphics engine 712 generates binocular video data that includes left-view video data of one or more virtual objects as viewed by a left eye of a user of the XR system and right-view video data of one or more virtual objects as viewed by a right eye of the user of the XR system. An LED display driver receives the binocular video data and generates a left (first) LED array control signal based on the left-view video data, the left (first) LED array control signal causing left (first) one or more LEDs of a left (first) LED array of the left (first) LED near-eye display element to be energized in a left (first) order. The LED display driver also generates a right (second) LED array control signal based on the right-view video data, the right (second) LED array control signal causing right (second) one or more LEDs of a right (second) LED array of the right (second) LED near-eye display element to be energized in a right (second) order. The LED display driver generates a left (first) synchronous motor control signal based on the left-view video data, the left (first) synchronous motor control signal causing left (first) one or more motors of the left (first) LED near-eye display element to be powered in synchronization with the energization of the left (first) one or more LEDs. The LED display driver also generates a right (second) synchronous motor control signal based on the right-view video data, the right (second) synchronous motor control signal causing right (second) one or more motors of the right (second) LED near-eye display element to be powered in synchronization with the energization of the right (second) one or more LEDs. The LED driver simultaneously transmits the left (first) LED array control signal and the right (second) LED array control signal to the left (first) one or more LED arrays and the right (second) one or more LED arrays, respectively. The LED display driver also simultaneously transmits the left (first) synchronous motor control signal to the left (first) one or more motors and the right (second) synchronous motor control signal to the right (second) one or more motors, respectively. The simultaneous transmission of the left (first) LED array control signal and the right (second) LED array control signal and the left (first) synchronous motor control signal and the right (second) synchronous motor control signal causes the left (first) LED near-eye display element and the right (second) LED near-eye display element to generate a display provided to the user of the XR system.

[0070] In some examples, an XR display system includes an LED near-eye display element spanning a user's left and right eyes. The LED near-eye display element includes a left (first) viewing sector and a right (second) viewing sector of a circular sweep region of an LED array of the LED near-eye display element. A graphics engine 712 generates binocular video data that includes left-view video data of one or more virtual objects of an XR experience as viewed by a left eye of a user of the XR system and right-view video data of the one or more virtual objects as viewed by a right eye of the user of the XR system. An LED display driver receives the binocular video data and generates a left (first) LED array control signal based on the left-view video data, the left (first) LED array control signal causing one or more LEDs of the LED array to be energized in a left (first) order when one or more LEDs are located in the left (first) viewing sector of the circular sweep region. The LED display driver also generates a right (second) LED array control signal based on the right-view video data, the right (second) LED array control signal causing one or more LEDs of the LED array to be energized when one or more LEDs are located in the right (second) viewing sector. The LED display driver generates a synchronous motor control signal based on the binocular video data, the synchronous motor control signal causing one or more motors of the LED near-eye display element to be powered in synchronization with the energization of the one or more LEDs. The LED display driver transmits the left (first) LED array control signal to the one or more LEDs when one or more LEDs are located in the left (first) viewing sector and transmits the right (second) LED array control signal to the one or more LEDs when one or more LEDs are located in the right (second) viewing sector. The LED display driver also simultaneously transmits the synchronous motor control signal to the one or more motors. The simultaneous transmission of the left (first) and right (second) LED array control signals and the synchronous motor control signal causes the LED near-eye display element to produce a display provided to the user of the XR system.

[0071] In some examples, one or more LED array drivers 716 generate a magnetic field 742 that inductively couples to one or more LED array power receiver circuits 746 to power one or more LED array control signal receivers 744 and one or more LED arrays 732.

[0072] In some examples, one or more LED array control signal receivers 744 and one or more LED arrays 732 are powered by an electromechanical coupling such as a slip ring.

[0073] In some examples, one or more LED array drivers 716 transmit LED array control signals 722 to one or more LED array control signal receivers 744 using a wireless communication protocol such as, but not limited to, Bluetooth.

[0074] In some examples, one or more LED array drivers 716 transmit LED array control signals 722 to one or more LED array control signal receivers 744 via an electromechanical coupling such as a slip ring.

[0075] Figure 8 is an illustrative representation of a machine 800 within which instructions 810 (e.g., software, program, application, applet, app, or other executable code) can be executed to cause the machine 800 to perform any one or more of the methods discussed herein. The machine 800 can be used as a computer 120 of a head-mounted XR system 100 of an AR system, for example Figure 1A The instructions 810 can cause the machine 800 to perform any one or more of the methods or processes described herein, for example. The instructions 810 transform the general, unprogrammed machine 800 into a particular machine 800 programmed to perform the described and illustrated functions in the described manner. The machine 800 can operate as a stand-alone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 800 can operate in a server-client network environment as a server machine or a client machine, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 800, in combination with other components of the AR system, can be used as, but not limited to: a server, a client, a computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular phone, a smartphone, a mobile device, a head-mounted device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing the instructions 810 specifying actions to be taken by the machine 800. Further, although a single machine 800 is shown, the term "machine" can also be regarded as including a collection of machines that individually or jointly execute the instructions 810 to perform any one or more of the methods discussed herein.

[0076] Machine 800 may include a processor 802, a memory 804, and an I / O device interface 806, which may be configured to communicate with each other via a bus 844. In an example, the processor 802 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 808 and a processor 812 that execute instructions 810. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although Figure 8 multiple processors 802 are shown, machine 800 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0077] Memory 804 includes a main memory 814, a static memory 816, and a storage unit 818, all of which are accessible by the processor 802 via the bus 844. The main memory 804, the static memory 816, and the storage unit 818 store instructions 810 that implement any one or more of the methods or functions described herein. The instructions 810 may also reside, in whole or in part, within the main memory 814, within the static memory 816, within the non-transitory machine-readable medium 820 within the storage unit 818, within one or more of the processors 802 (e.g., within the cache memory of the processor), or within any suitable combination thereof during execution by the machine 800.

[0078] The I / O device interface 806 couples the machine 800 to the I / O device 846. One or more of the I / O devices 846 may be components of the machine 800 or may be separate devices. The I / O device interface 806 may include various interfaces to the I / O device 846 that the machine 800 may use to receive input, provide output, generate output, send information, exchange information, capture measurements, etc. The specific I / O device interface 806 included in a particular machine will depend on the type of the machine. It should be understood that the I / O device interface 806, the I / O device 846 may include Figure 8Many other components not shown. In various examples, the I / O device interface 806 can include an output component interface 828 and an input component interface 832. The output component interface 828 can include an interface to visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), tactile components (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The input component interface 832 can include an interface to alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), pointing-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), tactile input components (e.g., a physical button, a touch screen that provides the location and / or force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), etc.

[0079] In additional examples, the I / O device interface 806 can include: a biometric component interface 834, a motion component interface 836, an environmental component interface 838, or a location component interface 840, as well as various other component interfaces. For example, the biometric component interface 834 can include an interface to components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biometric signals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component interface 836 can include an interface to an inertial measurement unit (IMU), an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotational sensor component (e.g., a gyroscope), etc. The environmental component interface 838 can include, for example, an interface to an illuminance sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect the ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects the concentration of a hazardous gas for safety or measures pollutants in the atmosphere), or other components that can provide an indication, measurement, or signal associated with the surrounding real-world scene. The location component interface 840 includes an interface to location sensor components (e.g., a global positioning system (GPS) receiver component and / or an inertial measurement unit (IMU)), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0080] A variety of techniques can be used to implement communication. The I / O device interface 806 also includes a communication component interface 842, which is operable to couple the machine 800 to the network 822 or the device 824 via the couplings 830 and 826, respectively. For example, the communication component interface 842 may include an interface to a network interface component that interfaces with the network 822 or another suitable device. In another example, the communication component interface 842 may include an interface to a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, components (e.g., low power consumption), components, and other communication components for providing communication via other modalities. The device 824 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0081] In addition, the communication component interface 842 may include an interface to a component operable to detect an identifier. For example, the communication component interface 842 may include an interface to a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., for detecting optical sensors for the following: one-dimensional barcodes, such as Universal Product Code (UPC) barcodes; multi-dimensional barcodes, such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Hypercode, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). Additionally, various information can be derived via the communication component interface 842, such as location via Internet Protocol (IP) geolocation, location via signal triangulation, location via detecting an NFC beacon signal that can indicate a specific location, etc.

[0082] Various memories (e.g., the memory 804, main memory 814, static memory 816, and / or the memory of the processor 802) and / or storage units 818 may store one or more sets of instructions and data structures (e.g., software) that implement any of the methods or functions described herein or that are used by the methods or functions. These instructions (e.g., instructions 810), when executed by the processor 802, cause various operations to implement the disclosed examples.

[0083] Instructions 810 can be sent or received over network 822 via a network interface device (e.g., a network interface component included in communication component interface 842), using a transmission medium and using any one of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 810 can be sent or received using a transmission medium via a coupling 826 to device 824 (e.g., a peer-to-peer coupling).

[0084] Figure 9 FIG. 900 is an architecture diagram showing a software architecture 904 that can be installed on any one or more of the devices described herein. The software architecture 904 is supported by hardware such as a machine 902 that includes a processor 920, a memory 926, and an I / O component interface 938. In this example, the software architecture 904 can be conceptualized as a stack of layers, where each layer provides a specific function. The software architecture 904 includes layers such as an operating system 912, libraries 908, frameworks 910, and applications 906. In operation, an application 906 activates an API call 950 through the software stack and receives a message 952 in response to the API call 950.

[0085] The operating system 912 manages hardware resources and provides common services. The operating system 912 includes, for example: a kernel 914, services 916, and drivers 922. The kernel 914 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 914 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. The services 916 can provide other common services to other software layers. The drivers 922 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 922 can include an LED display driver, a camera device driver, or a low-power driver, a flash drive, a serial communication driver (e.g., a Universal Serial Bus (USB) driver), a driver, an audio driver, a power management driver, etc.

[0086] Library 908 provides low-level common infrastructure used by application 906. Library 908 may include system library 918 (e.g., C standard library), which provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, library 908 may include API library 924, such as media libraries (e.g., libraries for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), High Efficiency Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., OpenGL framework for presenting 2D and 3D graphical content on a display, GLMotif for implementing user interfaces), image feature extraction libraries (e.g., OpenIMAJ), database libraries (e.g., SQLite providing various relational database functions), web libraries (e.g., WebKit providing web browsing functions), etc. Library 908 may also include various other libraries 928 to provide many other APIs to application 906.

[0087] Framework 910 provides high-level common infrastructure used by application 906. For example, framework 910 provides various Graphical User Interface (GUI) functions, advanced resource management, and advanced location services. Framework 910 may provide a wide range of other APIs that can be used by application 906, some of which may be specific to a particular operating system or platform.

[0088] In an example, application 906 may include a home application 936, a contacts application 930, a browser application 932, a book reader application 934, a location application 942, a media application 944, a messaging application 946, a gaming application 948, and a wide variety of other applications such as third-party applications 940 and XR applications 954. Application 906 is a program that executes functions defined in the program. One or more of the applications 906 can be created using various programming languages and constructed in various ways, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, a third-party application 940 (e.g., an application developed using an ANDROID TM or IOS TM software development kit (SDK)) can be an application on platforms such as IOS TM 、ANDROID TM 、 Mobile software running on the mobile operating system of a Phone or another mobile operating system. In this example, the third-party application 940 can activate the API call 950 provided by the operating system 912 to facilitate the functions described herein.

[0089] Figure 10 is a block diagram showing a networked system 1000 including details of a head-mounted XR system 100 according to some examples. The networked system 1000 includes a head-mounted XR system 100, a mobile computing system 1026, and a server system 1032. The mobile computing system 1026 can be a smart phone, a tablet, a phablet, a laptop computer, an access point, or any other such device capable of connecting to the head-mounted XR system 100 using a low-power wireless connection 1036 and / or a high-speed wireless connection 1034. The mobile computing system 1026 is connected to the server system 1032 via a network 1030. The network 1030 can include any combination of wired and wireless connections. The server system 1032 can be one or more computing devices that are part of a service or network computing system. Any elements of the mobile computing system 1026, as well as the server system 1032 and the network 1030, can be implemented using Figure 9 and Figure 8 the software architecture 904 or the details of the machine 800 described in.

[0090] The head-mounted XR system 100 includes a data processor 1002, a display 1010, one or more camera devices 1008, and additional input / output elements 1016. The input / output elements 1016 can include a microphone, an audio speaker, a biometric sensor, an additional sensor, or an additional display element integrated with the data processor 1002. Examples of the input / output elements 1016 are further discussed with respect to Figure 9 and Figure 8 For example, the input / output elements 1016 can include any of the I / O device interfaces 806 including an output component interface 828, a motion component interface 836, etc. Examples of the display 1010 are discussed in Figure 1B In the specific examples described herein, the display 1010 includes displays for the user's left and right eyes.

[0091] The data processor 1002 includes an image processor 1006 (e.g., a video processor), a GPU, and a display driver 1038, a tracking component 1040, an interface 1012, a low-power circuitry 1004, and a high-speed circuitry 1020. The components of the data processor 1002 are interconnected by a bus 1042.

[0092] Interface 1012 refers to any source of user commands provided to data processor 1002. In one or more examples, interface 1012 is a physical button that, when pressed, sends a user input signal from interface 1012 to low-power processor 1014. Low-power processor 1014 may process the pressing and then immediate release of such a button as a request to capture a single image, and vice versa. Low-power processor 1014 may process the pressing of such a button for a first time period as a request to capture video data while the button is pressed and to stop video capture when the button is released, where the video captured while the button is pressed is stored as a single video file. Alternatively, pressing the button for a long period of time may capture a still image. In some examples, interface 1012 may be any mechanical switch or physical interface capable of accepting user input associated with a data request from imaging device 1008. In other examples, interface 1012 may have a software component or may be associated with commands received wirelessly from another source such as mobile computing system 1026.

[0093] Image processor 1006 includes circuitry for receiving signals from imaging device 1008 and processing those signals from imaging device 1008 into a format suitable for storage in memory 1024 or suitable for transmission to mobile computing system 1026. In one or more examples, image processor 1006 (e.g., a video processor) includes a microprocessor integrated circuit (IC) customized for processing sensor data from imaging device 1008, and volatile memory used by the microprocessor in operation.

[0094] Low-power circuitry 1004 includes low-power processor 1014 and low-power wireless circuitry 1018. These elements of low-power circuitry 1004 may be implemented as separate elements or may be implemented on a single IC as part of a single system-on-chip. Low-power processor 1014 includes logic for managing other elements of the head-mounted XR system 100. As described above, for example, low-power processor 1014 may accept user input signals from interface 1012. Low-power processor 1014 may also be configured to receive input signals or command communications from mobile computing system 1026 via low-power wireless connection 1036. Low-power wireless circuitry 1018 includes circuit elements for implementing a low-power wireless communication system. Also known as Bluetooth TM Low-power Bluetooth TM Smart is a standard implementation of a low-power wireless communication system that may be used to implement low-power wireless circuitry 1018. In other examples, other low-power communication systems may be used.

[0095] The high-speed circuit system 1020 includes a high-speed processor 1022, a memory 1024, and a high-speed wireless circuit system 1028. The high-speed processor 1022 can be any processor capable of managing high-speed communication and operation for any general computing system for the data processor 1002. The high-speed processor 1022 includes processing resources for managing high-speed data transmission on the high-speed wireless connection 1034 using the high-speed wireless circuit system 1028. In some examples, the high-speed processor 1022 executes an operating system such as the LINUX operating system or other such operating systems such as Figure 9 the operating system 912. In addition to any other responsibilities, the high-speed processor 1022 that executes the software architecture for the data processor 1002 is also used to manage data transmission with the high-speed wireless circuit system 1028. In some examples, the high-speed wireless circuit system 1028 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, which is also referred to herein as Wi-Fi. In other examples, other high-speed communication standards can be implemented via the high-speed wireless circuit system 1028.

[0096] The memory 1024 includes any storage device capable of storing camera device data generated by the camera device 1008 and the image processor 1006. Although the memory 1024 is shown as integrated with the high-speed circuit system 1020, in other examples, the memory 1024 can be a separate stand-alone element of the data processor 1002. In some such examples, electrical wiring can provide a connection from the image processor 1006 or the low-power processor 1014 to the memory 1024 through a chip including the high-speed processor 1022. In other examples, the high-speed processor 1022 can manage the addressing of the memory 1024 such that the low-power processor 1014 will initiate the high-speed processor 1022 at any time when a read operation or a write operation involving the memory 1024 is desired.

[0097] The tracking component 1040 estimates the pose of the head-mounted XR system 100. For example, the tracking component 1040 uses image data and associated inertial data and GPS data from the camera device 1008 and the position component interface 840 to track the position and determine the pose of the head-mounted XR system 100 relative to a reference frame (e.g., a real-world scene environment). The tracking component 1040 continuously collects and uses updated sensor data describing the movement of the head-mounted XR system 100 to determine the updated three-dimensional pose of the head-mounted XR system 100, which indicates changes in the relative position and orientation with respect to physical objects in the real-world scene environment. The tracking component 1040 allows the head-mounted XR system 100 to visually place virtual objects relative to physical objects within the user's field of view via the display 1010.

[0098] The GPU and display driver 1038 can use the pose of the head - worn XR system 100 to generate frames of virtual content or other content to be presented on the display 1010 when the head - worn XR system 100 operates in a traditional augmented reality mode. In this mode, the GPU and display driver 1038 generate updated frames of virtual content based on the updated three - dimensional pose of the head - worn XR system 100, which reflects changes in the position and orientation of the user relative to physical objects in the user's real - world scene environment.

[0099] One or more of the functions or operations described herein can also be performed in an application residing on the head - worn XR system 100 or the mobile computing system 1026 or on a remote server. For example, one or more of the functions or operations described herein can be performed by one of the applications 906 such as the messaging application 946.

[0100] Figure 11 is a block diagram showing an example interaction system 1100 for facilitating interactions on a network (e.g., exchanging text messages, making text, audio, and video calls, or playing games). The interaction system 1100 includes a plurality of computing systems 1102, and each of the plurality of computing systems 1102 hosts a plurality of applications including an interaction client 1104 and other applications 1106. Each interaction client 1104 is communicatively coupled via one or more communication networks including a network 1108 (e.g., the Internet) to other instances of the interaction client 1104 (e.g., hosted on corresponding other computing systems 1102), an interaction server system 1110, and a third - party server 1112). The interaction client 1104 can also communicate with the locally - hosted applications 1106 using an application programming interface (API).

[0101] Each computing system 1102 can include one or more user devices communicatively connected to exchange data and messages, such as a mobile device 1114, a head - worn XR system 1116, and a computer client device 1118.

[0102] The interaction client 1104 interacts via the network 1108 with other interaction clients 1104 and with the interaction server system 1110. The data exchanged between the interaction clients 1104 (e.g., interaction 1120) and between the interaction client 1104 and the interaction server system 1110 includes functions (e.g., commands to activate functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0103] The interaction server system 1110 provides server - side functionality to the interaction client 1104 via the network 1108. Although certain functions of the interaction system 1100 are described herein as being performed by the interaction client 1104 or by the interaction server system 1110, the location of certain functions within the interaction client 1104 or within the interaction server system 1110 can be a design choice. For example, it may be technically preferable to initially deploy a particular technology and functionality within the interaction server system 1110, but later migrate the technology and functionality to the interaction client 1104 of the computing system 1102 that has sufficient processing power.

[0104] The interaction server system 1110 supports various services and operations provided to the interaction client 1104. Such operations include sending data to the interaction client 1104, receiving data from the interaction client 1104, and processing data generated by the interaction client 1104. The data can include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the interaction system 1100 is activated and controlled through functions available via the user interface (UI) of the interaction client 1104.

[0105] Turning now specifically to the interaction server system 1110, an application programming interface (API) server 1122 is coupled to and provides a programming interface for the interaction server 1124, making the functionality of the interaction server 1124 accessible to the interaction client 1104, other applications 1106, and third - party servers 1112. The interaction server 1124 is communicatively coupled to a database server 1126, thereby facilitating access to a database 1128 that stores data associated with interactions processed by the interaction server 1124. Similarly, a web server 1130 is coupled to the interaction server 1124 and provides a web - based interface to the interaction server 1124. To this end, the web server 1130 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0106] The Application Programming Interface (API) server 1122 receives and sends interaction data (e.g., commands and message payloads) between the interaction server 1124 and the computing system 1102 (and e.g., interaction client 1104 and other applications 1106) and the third-party server 1112. Specifically, the Application Programming Interface (API) server 1122 provides a set of interfaces (e.g., routines and protocols) that the interaction client 1104 and other applications 1106 can call or query to activate the functions of the interaction server 1124. The Application Programming Interface (API) server 1122 exposes various functions supported by the interaction server 1124, including account registration; login functionality; sending interaction data from a specific interaction client 1104 to another interaction client 1104 via the interaction server 1124; transmitting media files (e.g., images or videos) from the interaction client 1104 to the interaction server 1124; setting a collection of media data (e.g., a story); retrieving a friend list of a user of the computing system 1102; retrieving messages and content; adding and deleting entities (e.g., friends) for an entity graph (e.g., a social graph); locating friends within the social graph; and opening application events (e.g., related to the interaction client 1104).

[0107] Returning to the interaction client 1104, the features and functions of external resources (e.g., linked applications 1106 or applets) are available to the user via the interface of the interaction client 1104. In this context, "external" refers to the fact that the application 1106 or applet is outside of the interaction client 1104. Although external resources are typically provided by third parties, they can also be provided by the creator or provider of the interaction client 1104. The interaction client 1104 receives a user selection of options for initiating or accessing the features of such external resources. The external resources can be an application 1106 (e.g., a "native app") installed on the computing system 1102, or a scaled-down version of an application (e.g., an "applet") hosted on the computing system 1102 or located remotely from the computing system 1102 (e.g., on the third-party server 1112). The scaled-down version of the application includes a subset of the features and functions of the application (e.g., the full-scale, native version of the application) and is implemented using a markup language document. In some examples, the scaled-down version of the application (e.g., an "applet") is a web-based markup language version of the application and is embedded within the interaction client 1104. In addition to using a markup language document (e.g., a.*ml file), the applet can include a scripting language (e.g., a.*js file or a.json file) and a style sheet (e.g., a.*ss file).

[0108] In response to receiving a user selection of an option for a feature to initiate or access an external resource, the interactive client 1104 determines whether the selected external resource is a web-based external resource or a locally installed application 1106. In some cases, an application 1106 locally installed on the computing system 1102 can be launched independently of the interactive client 1104 and, for example, separately from the interactive client 1104 by selecting an icon corresponding to the application 1106 on the home screen of the computing system 1102. A scaled-down version of such an application can be launched or accessed via the interactive client 1104, and in some examples, no part of the scaled-down application can be accessed outside of the interactive client 1104 or only a limited portion of the scaled-down application can be accessed outside of the interactive client 1104. The scaled-down application can be launched by receiving, for example, a markup language document associated with the scaled-down application from a third-party server 1112 by the interactive client 1104 and processing such a document.

[0109] In response to determining that the external resource is a locally installed application 1106, the interactive client 1104 instructs the computing system 1102 to launch the external resource by executing locally stored code corresponding to the external resource. In response to determining that the external resource is a web-based resource, the interactive client 1104 communicates with, for example, a third-party server 1112 to obtain a markup language document corresponding to the selected external resource. The interactive client 1104 then processes the obtained markup language document to render the web-based external resource within the user interface of the interactive client 1104.

[0110] The interactive client 1104 can notify a user of the computing system 1102 or other users related to such a user (e.g., "friends") of activities taking place in one or more external resources. For example, the interactive client 1104 can provide notifications to participants in a conversation (e.g., a chat session) within the interactive client 1104 regarding the current or recent use of an external resource by one or more members of a group of users. One or more users can be invited to join an active external resource or launch an external resource that was recently used but is currently inactive (within the group of friends). An external resource can provide the ability for participants in a chat session, each using a respective interactive client 1104, to share items, conditions, statuses, or locations within the external resource with one or more members of a group of users. A shared item can be an interactive chat card that members of the chat can interact with, for example, to launch a corresponding external resource, view specific information within the external resource, or be taken to a specific location or status within the external resource. Within a given external resource, response messages can be sent to users on the interactive client 1104. The external resource can selectively include different media items in the response based on the current context of the external resource.

[0111] The interactive client 1104 can present a list of available external resources (e.g., applications 1106 or applets) to the user to initiate or access a given external resource. The list can be presented in the form of a context-sensitive menu. For example, the icons of different applications (or applets) representing the application 1106 (or applet) can vary based on how the menu is initiated by the user (e.g., from a conversational interface or from a non-conversational interface).

[0112] "Carrier signal" means any non-tangible medium capable of storing, encoding, or carrying instructions executable by a machine, and includes digital or analog communication signals or other non-tangible media facilitating the communication of such instructions. The instructions can be sent or received via a network interface device over a network using a transmission medium.

[0113] "Client device" means any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. The client device can be, but is not limited to, a mobile phone, desktop computer, laptop computer, portable digital assistant (PDA), smart phone, tablet, ultrabook, netbook, notebook computer, multi-processor system, microprocessor-based or programmable consumer electronics, game console, set-top box, or any other communication device that a user can use to access a network.

[0114] "Communication network" means one or more portions of a network, which can be an ad-hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), plain old telephone service (POTS) network, cellular telephone network, wireless network, network, another type of network, or a combination of two or more such networks. For example, a network or a portion of a network can include a wireless network or a cellular network, and the coupling can be a code division multiple access (CDMA) connection, global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling can implement any one of various types of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), evolved data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), worldwide interoperability for microwave access (WiMAX), long term evolution (LTE) standard, other data transmission technologies defined by various standards-setting organizations, other long-distance protocols, or other data transmission technologies.

[0115] "Machine-readable medium" means both machine storage media and transmission media. Thus, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "machine-readable medium", and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.

[0116] "Machine storage medium" means a single or multiple storage devices and / or media (e.g., a centralized or distributed database, and / or associated cache and server) that store executable instructions, routines, and / or data. The term includes, but is not limited to, solid-state memory as well as optical and magnetic media, including memory internal or external to a processor. Specific examples of machine storage media, computer storage media, and / or device storage media include, by way of example, non-volatile memory including semiconductor storage devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium".

[0117] "Processor" means any circuit or virtual circuit (a physical circuit simulated by logic running on an actual processor) that manipulates data values according to control signals (e.g., "commands", "opcodes", "machine codes", etc.) and produces associated output signals that are applied to operate a machine. For example, a processor can be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor can also be a multi-core processor having two or more independent processors (sometimes called "cores") that can execute instructions simultaneously.

[0118] "Signal medium" means any non - tangible medium that can store, encode, or carry instructions executed by a machine and includes digital or analog communication signals or other non - tangible media that facilitate the communication of software or data. The term "signal medium" shall be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a way as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0119] Changes and modifications can be made to the disclosed examples without departing from the scope of the disclosure. These and other changes or modifications are intended to be included within the scope of the disclosure as expressed in the appended claims.

Claims

1. An extended reality (XR) display system, comprising: A light-emitting diode (LED) display driver; An LED near-eye display element operatively coupled to the LED display driver, the LED near-eye display element including one or more motors and an LED array operably connected to the one or more motors; One or more processors; And A memory operably connected to the one or more processors, the memory storing executable instructions that, when executed by a machine, cause the XR display system to perform operations, the operations including: Receiving, by the LED display driver, video data of an XR application, the video data including rendered virtual objects of an XR experience provided by the XR application; Generating, by the LED display driver, an LED array control signal based on the video data, the LED array control signal causing one or more LEDs of the LED array to be energized in sequence; Generating, by the LED display driver, a synchronous motor control signal based on the video data, the synchronous motor control signal causing the one or more motors to be powered in synchronization with the energization of the one or more LEDs; and Simultaneously transmitting, by the LED display driver, the LED array control signal to the LED array and the synchronous motor control signal to the one or more motors, causing the LED near-eye display element to display the rendered virtual objects to a user of the XR system.

2. The XR display system according to claim 1, wherein, The one or more motors cause the LED array to sweep across a circular sweep area.

3. The XR display system according to claim 1, wherein, The one or more motors cause the LED array to sweep across a sector of the circular sweep area.

4. The XR display system according to claim 1, wherein, The one or more motors cause the LED array to sweep across a rectangular sweep area.

5. The XR display system according to claim 4, wherein, The LED array is operably coupled to the one or more motors by one or more traction belts.

6. The XR display system according to claim 5, wherein, The LED array is operably coupled to the one or more traction belts by one or more rotatable connectors.

7. The XR display system according to claim 1, wherein, The LED array of the LED near-eye display element is operably coupled to the one or more motors by a magnetic ring gear and one or more magnetic pinions.

8. The XR display system according to claim 1, wherein The LED array is powered by inductive coupling.

9. The XR display system according to claim 1, wherein, The LED array is controlled via a wireless communication protocol.

10. The XR display system according to claim 1, wherein, The XR display system is a component of a head-mounted XR system.

11. An XR display system, comprising: An LED display driver; A first light-emitting diode (LED) near-eye display element operatively coupled to the LED display driver, the first LED near-eye display element including first one or more motors and a first LED array operably connected to the first one or more motors; A second LED near-eye display element operatively coupled to the LED display driver, the second LED near-eye display element including second one or more motors and a second LED array operably connected to the second one or more motors; One or more processors; And A memory operably connected to the one or more processors, the memory storing executable instructions that, when executed by a machine, cause the XR display system to perform operations, the operations including: Receiving binocular video data of an XR application by the LED display driver, the binocular video data including rendered virtual objects of an XR experience provided by the XR application; Generating, by the LED display driver, an LED array control signal based on the binocular video data, the LED array control signal causing a first one or more LEDs of the first LED array to be energized in a first order and causing a second one or more LEDs of the second LED array to be energized in a second order; Generating, by the LED display driver, a synchronous motor control signal based on the binocular video data, the synchronous motor control signal causing the first one or more motors to be powered synchronously with the energization of the first one or more LEDs and causing the second one or more motors to be powered synchronously with the energization of the second one or more LEDs; and Simultaneously transmitting, by the LED display driver, the LED array control signal to the first one or more LED arrays and the second one or more LED arrays and transmitting the synchronous motor control signal to the first one or more motors and the second one or more motors, causing the first LED near-eye display element and the second LED near-eye display element to display the rendered virtual objects to a user of the XR system.

12. The XR display system according to claim 11, wherein, The first one or more motors cause the first LED array to sweep through a first circular sweep area, and the second one or more motors cause the second LED array to sweep through a second circular sweep area.

13. The XR display system according to claim 11, wherein, The first one or more motors cause the first LED array to sweep through a first sector of a circular sweep area, and the second one or more motors cause the second LED array to sweep through a second sector of the circular sweep area.

14. The XR display system according to claim 11, wherein, The first one or more motors cause the first LED array to sweep through a first rectangular sweep area, and the second one or more motors cause the second LED array to sweep through a second rectangular sweep area.

15. The XR display system according to claim 14, wherein, The first LED array is operably connected to the first one or more motors by a first one or more traction belts, and the second LED array is operably connected to the second one or more motors by a second one or more traction belts.

16. The XR display system according to claim 15, wherein, The first LED array is connected to the first one or more traction belts by a first one or more rotatable connectors, and the second LED array is connected to the second one or more traction belts by a second one or more rotatable connectors.

17. The XR display system according to claim 11, wherein, The first LED array is operably connected to the first one or more motors via a first magnetic ring gear and a first one or more magnetic pinions, and the second LED array is operably connected to the second one or more motors via a second magnetic ring gear and a second one or more magnetic pinions.

18. The XR display system according to claim 11, wherein, The LED array is powered by inductive coupling.

19. The XR display system according to claim 11, wherein, The XR display system is a component of a head-mounted XR system.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by a machine, cause the machine to: Receive binocular video data of an XR application, the binocular video data including rendered virtual objects of an XR experience provided by the XR application; Generate an LED array control signal based on the binocular video data, the LED array control signal causing a first one or more LEDs to be energized in a first sequence, the first LEDs including a first LED array operably connected to a first one or more motors, and the LED array control signal causing a second one or more LEDs to be energized in a second sequence, the second LEDs including a second LED array operably connected to a second one or more motors; Generate a synchronous motor control signal based on the binocular video data, the synchronous motor control signal causing the first one or more motors to be powered in synchronization with the energization of the first one or more LEDs, and causing the second one or more motors to be powered in synchronization with the energization of the second one or more LEDs; And Simultaneously transmit the LED array control signal to the first one or more LED arrays and the second one or more LED arrays and transmit the synchronous motor control signal to the first one or more motors and the second one or more motors, causing the first LED near-eye display element and the second LED near-eye display element to display the rendered virtual objects to a user of the XR system.

Images