Opacity control for augmented reality devices

Abstract

An augmented reality (AR) eyewear device has a lens system that includes an optical shielding mechanism that enables the lens system to switch between a conventional see-through state and an opaque state in which the lens system shields or functionally blocks a wearer's field of view to an external environment. Such a shielding mechanism allows for extended use cases of AR glasses compared to conventional devices, for example: as a sleep mask; the displayed content, such as a movie or a sports event, can be watched under the background that the attention is not distracted visually rather than under the external environment; and / or starting a VR function.

Description

[0001] Priority claim

[0002] This application claims the benefit of priority to U.S. patent application Ser. No. 17 / 950,923, filed on Sep. 22, 2022, which is incorporated herein by reference in its entirety. Background Art

[0003] Head-mounted augmented reality (AR) devices (e.g., AR-enabled glasses) are typically implemented with a near-eye display carried by a transparent or translucent lens assembly, through which the surrounding environment (the "real world") is visible to the wearer. Such near-eye displays are themselves typically partially transparent, so that objects presented by the near-eye display (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) appear to the wearer to be superimposed on their real-world environment. The AR displays in such devices are typically stereoscopic, creating an illusion of depth and / or 3D position in the environment by displaying two slightly different images to the wearer's right and left eyes.

[0004] This is often referred to as "augmented reality," as distinguished from the experience provided by a head-mounted device that completely obscures the wearer's field of view and displays a virtual environment through which the wearer can appear to move or be moved, which is often referred to as "virtual reality" or VR. As used herein, unless the context indicates otherwise, the term "augmented reality" or "AR" refers to both augmented reality and virtual reality as conventionally understood.

[0005] In some examples, the AR device takes the form of AR glasses, which are eyewear devices (commonly referred to as smart glasses) that support electronic devices and are configured to be conventionally worn in the manner of corrective glasses or sunglasses. As with conventional non-smart glasses, a pair of AR glasses typically has a lens system provided by a left lens assembly and a right lens assembly, through which the surrounding environment is viewed during wear, wherein a near-eye AR display is typically inherently integrated into the glasses to present a visual image in a corresponding display area in the lens system. Some examples of such near-eye displays include a waveguide contained in a lens assembly to receive a light beam from a projector, but a variety of different suitable display mechanisms have been used in other cases. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] In the drawings, which are not necessarily drawn to scale, similar reference numerals may describe similar components in different views. To easily identify the discussion of any particular element or action, the highest digit or digits in the reference numeral refer to the figure number in which the element is first introduced. Some non-limiting examples are shown in the accompanying drawings, in which:

[0007] Figure 1 is a schematic three-dimensional illustration of an AR-enabled eyewear device in the form of AR glasses, which are smart glasses with inherent AR capabilities, according to one example.

[0008] Figure 2A Based on an example such as Figure 1 Schematic cross-sectional side view of AR glasses, which are shown as being set in AR mode, in which the lens system of the AR glasses is in a perspective state.

[0009] Figure 2B Based on an example Figure 2A The example AR glasses are shown as being set in VR mode, in which the lens system is in an opaque state.

[0010] Figure 3A FIG. 1 shows a diagram of the wearer's perspective during operation of the glasses in AR mode according to an example. Figure 2A The view of AR glasses.

[0011] Figure 3B According to an example, Figure 3A The corresponding view, AR glasses are set to Figure 2B VR mode.

[0012] Figure 4 is a diagram showing a method according to an example including Figure 1 Block diagram of the networking system of a wearable AR device.

[0013] Figure 5 is a diagrammatic representation of a networking environment in which the present disclosure may be deployed, according to some examples.

[0014] Figure 6 is a high-level flow chart schematically illustrating a method 600 for dynamic autonomous control of an operating mode of an AR-enabled head mounted device according to one example.

[0015] Figure 7 is a flow chart according to an example, which schematically illustrates a method for Figure 1 Flowchart 700 of dynamic autonomous control of operating modes of an AR-enabled eyewear device consistent with an example AR eyewear.

[0016] Fig. 8A is a schematic three-dimensional illustration of an eyewear device having a manually controllable optical shielding mechanism included in a lens system thereof, the lens system being set in a maximally transparent optical state according to an example.

[0017] Figure 8B is with Fig. 8A , the lens system is set in a partially transparent, darkened state.

[0018] Figure 8C is with Fig. 8A , the lens system is set in a maximally opaque light-shielding state.

[0019] Fig. 9 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein.

[0020] Fig.10 is a block diagram illustrating a software architecture in which examples may be implemented. DETAILED DESCRIPTION

[0021] One example of the present disclosure provides AR glasses, the lens system of which includes an optical shielding mechanism that enables the lens system to switch between a conventional see-through state and an opaque state, in which the lens system shields or functionally blocks the wearer's view of the external environment. Such a shielding mechanism allows for expanded use cases of AR glasses compared to conventional devices, such as: as a sleeping mask; viewing displayed content such as a movie or sporting event in a visually non-distracting background rather than in an external environment; and / or enabling VR functionality.

[0022] In the see-through state, the lens system is functionally transparent, meaning that it is transparent enough to allow the AR glasses to be used in the manner of corrective glasses or sunglasses. In the opaque state, the lens system is functionally opaque, also referred to herein as functionally opaque. Note that describing an object as "functionally opaque" or "functionally non-transparent" herein includes not only that the object can be opaque (i.e., not transmissive to visible light, and therefore neither transparent nor translucent, and for ease of understanding, this state is sometimes referred to herein as completely opaque), but also that the object can be translucent (i.e., transmitting some light but not enough to be functionally transparent, or causing sufficient diffusion to prevent the perception of different images).

[0023] In some examples, the shielding mechanism enables the AR glasses to switch to an occluded state, in which the lens system is completely opaque, thereby completely blocking the view of the environment. It should be understood that, as used herein, the occluded state and the opaque state are not mutually exclusive, but the occluded state is a specific, more limited instance of the opaque state. In other words, in all cases where the lens system is in the occluded state, it is also in the opaque state, but the lens system can be in an opaque state without being in the occluded state in some cases (e.g., being translucent or having an opacity level of 98%, allowing 2% of visible light to pass).

[0024] One example where a wearer may advantageously choose to use the AR glasses in the opaque state is for viewing non-AR visual media, such as movies or sporting events. Thus, the combination of the near-eye display and the opaque state effectively enables the use of the AR glasses in a VR mode or a cinema mode, in which the wearer can, for example, sit down and watch a movie without being disturbed. It should be understood that in the see-through state, the near-eye display disposed in or on the lens system is arranged to be backgrounded or superimposed on the wearer's view of their environment provided by light passing through the lens assembly, in effect backlighting the near-eye display. However, in the opaque state, shielding such backlighting improves the perceived quality and brightness of the display, which has a much larger background in comparison. In addition, shielding the user from their surroundings in the opaque state greatly reduces visual distractions from the content presented to the wearer.

[0025] In other cases, when set to an opaque state when the near-eye display is not in use, the glasses effectively provide the functionality of a sleep mask, blocking ambient light to facilitate falling asleep or undisturbed rest. Note that some examples of the present disclosure provide integrated lens shielding mechanisms included in eyewear devices without integrated display functionality. In one example, the shielding mechanism for such non-AR and / or non-smart eyewear devices includes a dual polarizer shielding technology that is capable of dynamic adjustment, as further described below with reference to display-enabled devices.

[0026] In some examples, the shielding mechanism enables the opacity of the lens system to be graded or smoothly varied between a maximum transparent setting at the upper limit of lens clarity (i.e., at the upper limit of transparency or optical transmittance, and conversely at the lower limit of opacity) on the one hand and a maximum opaque setting at the darkest limit of the opaque state on the other hand. Such graded or continuous adjustability of the lens opacity allows for automatic or user-controlled darkening or lightening of the lens assembly, effectively changing the tint of the glasses to provide a sunglass function according to personal taste or in response to ambient light conditions.

[0027] In some examples that allow the shielding mechanism to be switched to the blocking state, the shielding mechanism includes two stacked polarizers that are movable relative to each other so as to allow the respective polarization axes to be caused to be perpendicular to each other. As is well known, two stacked polarizers whose polarization axes are perpendicular to each other are actually completely opaque, actually blocking all light from passing therethrough.

[0028] A "polarizer" is an optical element that allows light waves of a specific polarization to pass through, while blocking light waves of other polarizations. It can filter a beam of light of undefined or mixed polarization into a beam of light of well-defined polarization, i.e., polarized light. The polarization axis of the polarizer is defined by the direction of the electric field of the light wave along which the polarizer passes, so that the polarization axis is parallel to the plane in which the light wave thus filtered oscillates. In some examples, the polarizer is a pair of linear polarizers, and in other examples a pair of circular polarizers.

[0029] "Stacking" refers to the relative spatial arrangement of related elements (in the context of optical eyewear elements such as lenses, polarizers or filters) so that these related elements are more or less parallel to each other and aligned one behind the other along the user's line of sight, which are laterally oriented to the user's line of sight. Thus, the stacked optical elements are spaced one behind the other along the user's line of sight, so that the user's view of the surrounding environment is provided by light that passes through all elements in the stacked elements sequentially. The stacked elements do not need to be in close proximity to each other. Thus, for example, a lens assembly comprising two polarizers that together sandwich a transparent lens comprises a pair of stacked polarizers.

[0030] In some examples, the pair of polarizers is provided by a fixed polarizing lens and a movable polarizing lens or filter housed in an actuating bracket that allows it to rotate 90° relative to the fixed lens. In one example, the actuating bracket is provided by a circular bezel ring or frame extending circumferentially around the radial periphery of the circular polarizing filter. In some embodiments, control of the shielding mechanism is exclusively selective in response to user input. In different embodiments, various user control mechanisms can be employed individually or in combination.

[0031] Thus, for example, a user can selectively control the shielding mechanism to block or not block ambient light by providing an opacity control input, such as by pressing a control button on the glasses, by using a companion mobile phone application, by using voice control, and / or by using other manual interactions, such as tactile input to a touch screen side panel on the temple of the glasses and / or a series of tap inputs. In various other examples, other suitable input mechanisms may alternatively or additionally be employed.

[0032] Alternatively or additionally, some examples provide autonomous switching between a see-through state and an opaque state (also referred to as AR mode and VR mode, respectively, in some cases herein) based on a use case and / or sensor data and / or biometric data indicative of user behavior. For example, in some examples, an opacity controller included in the glasses is configured to monitor whether the wearer of the device has fallen asleep while in the see-through state, and if so, automatically switch the lens system to the opaque state by actuating rotation of an adjustable polarizing filter. In some examples, automatic switching of the optical state of the lens system is achieved using a trained machine learning model to identify when to perform an autonomous switch from an opaque state to a see-through state, and / or vice versa.

[0033] Accordingly, one aspect of the present disclosure provides a device comprising:

[0034] A lens system comprising a lens assembly;

[0035] an eyeglass body on which the lens assembly is mounted, the eyeglass body being configured for head-mounted wear, wherein during head-mounted wear, the lens assembly is supported in a suitable position to occupy the wearer's field of view so that the wearer's perspective view of the surrounding environment passes through the lens assembly;

[0036] a display mechanism configured to provide a near-eye display to a wearer that visually coincides with the lens assembly; and

[0037] A shielding mechanism included in the lens assembly and configured to selectively set the lens assembly between at least two different optical states, the at least two different optical states comprising:

[0038] a see-through state in which the lens assembly is functionally transparent such that the near-eye display is superimposed on a view of the surrounding environment provided by external light passing through the lens assembly; and

[0039] An opaque state in which the lens assembly is functionally opaque to ambient light, blocking ambient backlighting of the near-eye display.

[0040] In some embodiments, the lens system is provided by a pair of lens assemblies, each of which is held in front of a respective eye of the wearer during use, wherein the shielding mechanism comprises a respective optical shielding arrangement provided by a separate component of each of the lens assemblies. In some examples, the shielding mechanism is configured such that the opaque state of the lens assembly constitutes a blocked state in which the lens assembly is completely opaque, substantially blocking any ambient light from passing through the lens assembly. One such example is a pair of stacked polarizers whose polarization axes may be positioned orthogonal to each other.

[0041] In some examples, the shielding mechanism is configured to enable graded variation of the opacity of the lens assembly, thereby enabling controlled graded variation of the intensity of the ambient backlight for the near-eye display. One such example is provided by one of a pair of stacked polarizers including a polarizing filter that can be smoothly rotated relative to the other polarizer.

[0042] In some examples, the device includes one or more opacity control input channels that allow for selective user-controlled graded variation of the opacity of the lens assembly. Examples of such input channels include a manual input receiver (e.g., a button or touchpad) carried by the eyeglass body, and alternatively or additionally, a mechanical lever or control pad operable to control lens opacity.

[0043] In some examples, the shielding mechanism is configured so that the objective intensity of the near-eye display is not affected by changes in the opacity of the lens assembly. In some examples, this is achieved by positioning the near-eye display in front of the closer of a pair of polarizers, as viewed from the wearer's perspective. Ambient light is polarized by the other of the polarizers in use, and is then largely or completely blocked by the closer polarizer when in the opaque mode. The effect of such an arrangement is that a reduction in the opacity of the shielding mechanism increases the brightness contrast between the foreground and background of the display, thereby increasing the subjectively perceived brightness or intensity of the display.

[0044] Therefore, in some examples, the shielding mechanism includes a pair of stacked polarizers contained in the lens assembly, and the pair of polarizers can be selectively moved relative to one polarizer to modify the composite opacity of the pair of polarizers by changing the relative orientations of the corresponding polarization axes of the stacked polarizers, thereby achieving switching of the lens assembly between a see-through state and an opaque state. In some embodiments, the opaque state includes a blocked state.

[0045] In some embodiments, the pair of stacked polarizers is provided by a static polarizer fixed in position relative to the eyeglass body and a movable polarizer configured to be selectively movable relative to the eyeglass body about an axis of rotation that is substantially aligned with an operational viewing direction through the lens assembly. In some examples, the static polarizer is integrated with a primary optical element of the lens assembly, such as comprising a polarizing film attached to an outwardly directed primary face of a glass or polymer lens element that carries a near-eye display, and the movable polarizer is located on a front side of the primary optical element, away from a wearer of the device, so that the near-eye display is located in front of the movable polarizer in the wearer's field of view.

[0046] In some examples, the movable polarizer is manually adjustable in rotational orientation relative to the axis of rotation. In other examples, the shielding mechanism further includes an actuator (e.g., an electromechanical motor) configured to achieve driven modification of the rotational orientation of the movable polarizer relative to the static polarizer. In some such examples, the shielding mechanism further includes a selective control mechanism providing an opacity control input channel, the opacity control input channel being configured to receive a mode switching command provided by a user, and in response to the mode switching command, switching the lens assembly between an opaque state and a see-through state by movement of the movable polarizer driven by operation of the actuator.

[0047] In some examples, the device is configured to autonomously control the opacity of one or more lens assemblies via a shielding mechanism. In such examples, the device also includes an opacity controller provided by one or more computer processor devices housed in the eyeglass body, the one or more computer processor devices being configured to dynamically autonomously switch modes by performing operations including: continuously receiving current sensor data captured by one or more sensors contained in the eyeglass body; and autonomously switching the lens assembly from one of a see-through state and an opaque state to another state based at least in part on the current sensor data. In some such examples, the opacity controller is configured to provide automatic control of the shielding mechanism by performing operations including: when the lens assembly is in a first optical state in its optical state, identifying satisfaction of a predefined mode switching criterion applicable to the first optical state, and in response to satisfying the applicable mode switching criterion, automatically implementing the setting of the lens assembly from the first optical mode in the optical state of the lens assembly to a different second optical mode.

[0048] In one example, the switching criteria for automatically switching the lens assembly from the see-through state to the opaque state includes a combination of: (a) the wearer of the device is in a non-walking state, and (b) receiving a selection input to reproduce non-AR visual content through the near-eye display. Thus, when the wearer sits down and begins watching a movie, the glasses automatically switch to an occluded mode or state.

[0049] Alternatively or additionally, the mode switching criteria for switching the lens assembly from the see-through state to the opaque state includes recognizing that the wearer has entered a sleeping state. Thus, when the user falls asleep, the eyewear device automatically switches to a sleeping mode in which ambient light is blocked, i.e., a sleeping mask style.

[0050] Another aspect of the present disclosure provides a method of operating an eyewear device, the method comprising:

[0051] accessing operational data from an eyewear device having support electronics for an optical system capable of switching between different optical states, the different optical states comprising at least (a) a see-through state and (b) an opaque state in which a wearer's view of their environment is obstructed by a lens assembly, the operational data comprising a current optical state of the optical system and sensor data captured by one or more sensors integrated into the eyewear device;

[0052] When the lens assembly is in a first one of its optical states, identifying satisfaction of a predefined mode switching criterion applicable to the first optical state; and

[0053] In response to satisfying the mode switching criterion, setting of the lens assembly from the first optical state mode to a different second optical state mode of the optical state of the lens assembly is automatically effected.

[0054] The method may also include, when the lens assembly is in the see-through state, wherein the applicable predefined mode switching criteria includes identifying that the wearer has fallen asleep.

[0055] In some examples, autonomous mode switching functionality is provided by only one of the optical states (e.g., autonomously switching from opaque to see-through only), but in other examples, bidirectional autonomous mode switching is provided. In some examples, when the lens assembly is in the see-through state, the applicable predefined mode switching criteria include identifying that the wearer is in a non-walking state, such as sitting or lying down.

[0056] Another aspect of the present disclosure provides a non-transitory computer-readable storage medium comprising instructions that, when processed by a computer, configure the computer to perform operations comprising:

[0057] accessing operational data from an eyewear device having support electronics for an optical system capable of switching between different optical states, the different optical states comprising at least (a) a see-through state and (b) an opaque state in which a wearer's view of their environment is obstructed by a lens assembly, the operational data comprising a current optical state of the optical system and sensor data captured by one or more sensors integrated into the eyewear device;

[0058] When the lens assembly is in a first one of its optical states, identifying satisfaction of a predefined mode switching criterion applicable to the first optical state; and

[0059] In response to satisfying the mode switching criterion, setting of the lens assembly from the first optical state mode to a different second optical state mode of the optical state of the lens assembly is automatically effected.

[0060] Other technical features may be readily apparent to those skilled in the art from the following drawings, descriptions and claims.

[0061] Turning now to a more detailed description of specific example embodiments, Figure 1 An oblique front view of an eyewear device supporting an electronic device in the form of a pair of AR glasses 100 is shown, the AR glasses 100 having an integrated near-eye display mechanism 106 and an optical shielding mechanism 102 integrated in a lens system 104 of the AR glasses 100. The shielding mechanism 102 enables the lens system 104 to operate in at least two different operating modes, wherein the transmittance of the lens system 104 relative to ambient light is in different corresponding optical states. In this example, the different optical states of the shielding mechanism 102 include: (a) a see-through state, in which the lens system 104 is functionally transparent so that the AR glasses 100 can operate in an AR mode in which augmented reality functions are available; and (b) an opaque state, in which the lens system 104 is functionally opaque, including an occlusion state in this example, in which the wearer's view of his external environment is completely blocked or blocked.

[0062] As will be described in more detail below, use cases for the AR glasses 100 with the lens system 104 in the opaque state include a sleep mode in which the AR glasses 100 function as a form of a sleep mask, thereby facilitating rest or sleep at appropriate times during wear. Additionally, when the shielding mechanism 102 renders the lens system 104 opaque, visual media content may be played via a near-eye display. This mode of operation is sometimes referred to herein as a theater mode, in which, for example, the viewing experience of video content such as a movie or sporting event is greatly enhanced because the shielding mechanism 102 provides an opaque background to the near-eye display, thereby shielding away visual distractions that would otherwise be the visual background of the display, as is the case in AR mode. Additionally, the significantly darkened or blackened visual background provided by the shielding mechanism 102 in some examples increases the perceived brightness of the near-eye display, as compared to an AR mode in which the display is backlit by ambient light.

[0063] In this example, the AR glasses 100 are also configured to be able to operate in VR mode while the lens system 104 is in an opaque state, in this example, more or less completely blocking ambient light. The display mechanism 106 in VR mode displays virtual reality content set relative to the blocked background. It should be understood that in this example, the difference between theater mode and VR mode lies in the nature of the content presented by the display mechanism 106, where VR content typically uses stereo effects to provide a virtual environment with the appearance of depth. Note that the terms VR mode and AR mode are used herein to indicate that the AR glasses 100 are in a configuration or optical state for providing an AR experience and a VR experience, respectively, but not necessarily providing any display. Thus, for example, when the lens system 104 is in a see-through state of conventional glasses, for the purposes of this description, the AR glasses 100 are considered to be in AR mode, regardless of whether anything is presented via a near-eye display.

[0064] The body 108 of the AR glasses 100 includes a front piece or frame 112 and a pair of temples 114 hinged to the frame 112 for supporting the frame 112 in a suitable position on the user's face when the temples 114 are in an extended or wearable configuration, such as Figure 1 In some examples, the entire frame 112 can be formed from a single piece of material so as to have a unitary or one-piece construction. In this example embodiment, the frame 112 is at least partially provided by one or more substantially rigid molded components formed from a polymeric plastic material.

[0065] The lens system 104 is supported by the frame 112 in a typical eyeglass fashion so that, during wear, the lens system 104 is positioned just in front of the eye so that the wearer's field of view is substantially completely occupied by the lens system 104. During typical use and wear, the lens system 104 is transparent or translucent, and the wearer thus views their environment through the transmissive optical element that provides the lens system 104.

[0066] The lens system 104 includes a pair of bilateral optical elements housed by a frame 112. In other words, the lens system 104 appears to be provided by a pair of eyeglass lenses mounted on the frame 112, the pair of eyeglass lenses being in a corresponding pair of lens rims 116 located on the left and right sides of the frame 112. However, each of these "lenses" includes a composite multifunctional lens assembly 110 having a plurality of stacked and interconnected transmissive optical elements that not only provide the basic functions of a conventional eyeglass lens, but also additionally provide or implement integrated display capabilities and dynamic optical shielding functions. The lens assembly 110 is mounted on the frame in a corresponding lens rim 116 that forms a portion of the frame 112.

[0067] In this example, the lens assembly 110 is non-corrective in terms of collective effect, thereby allowing light in AR mode to pass through the lens assembly 110 substantially without distortion. In this example, when the AR glasses 100 are in AR mode, during normal operation in a see-through state, the lens assembly 110 is substantially completely transparent. In other examples, the AR glasses 100 are configured for outdoor use as sunglasses, such that the lens assembly 110 is translucent, filtering a significant portion of the ambient light passing therethrough to provide a particular tint, while the lens assembly 110 is functionally transparent in a conventional sunglass style. Note that in other examples, the lens assembly 110 can be configured to have different optical effects and / or functions than this example, at least in AR mode, such as optical correction or being shaped for special functions, such as ski goggles, etc.

[0068] In this example, the optical shielding mechanism 102 includes a corresponding pair of stacked polarizers in each lens assembly 110 that are movable relative to each other within the lens assembly 110 to change the orientation of their corresponding polarization axes. Because the construction and operation of the stacked polarizers will be most easily understood in a side view and from the perspective of the wearer, these aspects of the shielding mechanism 102 are described below with reference to FIG. 2A to FIG. 3B A more detailed description is given.

[0069] Nevertheless, about Figure 1 It should be briefly noted that in each lens assembly 110, the pair of polarizers is provided by a static polarizer (fixed in position relative to the frame 112 and the lens bezel 116) and a movable polarizer that is rotationally movable relative to the frame 112 (and therefore relative to the static polarizer) about a rotation axis 120 that is more or less perpendicular to the lens assembly 110, and therefore generally parallel to the user's viewing direction through the lens assembly 110 during wear. In the present example, the static polarizer is provided by a polarizing fixed lens 122, which is located closer to the viewer and rotationally fixed relative to the lens bezel 116. Note that the fixed lens 122 can be a hybrid or compound lens arrangement. In the present example, the polarizing function of the fixed lens 122 is provided by a polarizing film 202 (see FIG. 202 ) adhered to the front face of the fixed lens 122. Figure 2A )supply.

[0070] The movable polarizer in this example comprises a polarizing filter 124 mounted in front of a fixed lens 122 (i.e., mounted to one of the major faces farthest from the wearer) for controlled partial rotational movement relative to the fixed lens 122 about its rotational axis 120.

[0071] In this example, both the fixed lens 122 and the polarizing filter 124 are linear polarizers configured to confine the full-wave oscillations of light passing therethrough to a single common plane referred to as its polarization axis. The fixed lens 122 accordingly has a polarization axis 302 that is fixed relative to the frame 112, and the polarizing filter 124 has a polarization axis 304 that can be adjusted (e.g., see Figure 3A and Figure 3B ).

[0072] In the present example, the rotation range of the polarization filter 124 is at least 90°, thus allowing the polarization axes 302, 304 to be positioned at corresponding extremes relative to each other, in which case they are parallel ( Figure 3A ), in combination do not block more light than they would individually, or in the case where they are perpendicular to each other, in combination block or screen out all ambient light and thus set the lens assembly 110 to a fully opaque state. In this example, the polarizing filter 124 is additionally capable of being set at any relative angle between the two extremes. It should be understood that the proportion of light screened by the combined pair of polarizers varies smoothly with the relative angle, so that by positioning the polarizing filter 124 at a relative angle corresponding to the desired opacity level, any opacity value between fully opaque and the maximum transmission limit can be obtained. The optical shielding mechanism 102 of the lens assembly 110 is therefore configured to enable a graded variation in the opacity of the lens system 104.

[0073] The eyeglass body 108 includes a pair of ends 118 at opposite ends of the frame 112, and the ends 118 provide corresponding internal spaces in which at least a portion of the onboard electronic device 126 of the AR glasses 100 is accommodated. In this example, various electronic components are housed in one or both of the ends 118. Some of the electronic devices 126 are also housed in the frame 112 and the temples 114. In some embodiments, the frame 112 is formed from a single piece of material so as to have an integral or monolithic construction. In this example embodiment, each end 118 is formed from a separate molded plastic component. In other examples, the frame is metal.

[0074] Onboard electronics 126 may include one or more processors with memory, wireless communications circuitry, and a power source. Figure 4 As discussed, the electronic device 126 includes low-power circuitry, high-speed circuitry, and a display processor. Various other examples may include these elements in different configurations or integrated together in different ways. The electronic device 126 additionally includes an onboard battery or other suitable portable power supply. The onboard electronic device 126 may include a connector or port (not shown) suitable for charging the battery, a wireless receiver, transmitter or transceiver (not shown), or a combination of such devices.

[0075] The AR glasses 100 are camera-enabled, in this example, including a pair of cameras 128 mounted on the frame 112 and facing forward so as to be more or less aligned with the viewing direction of the wearer of the AR glasses 100. The cameras 128 are configured to capture digital photographs as well as digital video content. Although two cameras are depicted, other examples contemplate the use of a single or additional (i.e., more than two) cameras. In one or more examples, in addition to the cameras 128, the AR glasses 100 also include any number of input sensors or other input / output devices. In this example, the onboard sensors provided by the electronic device 126 include, but are not limited to, biometric sensors, position sensors, and motion sensors and motion sensors.

[0076] The AR glasses 100 also include one or more input and output mechanisms that allow communication with the onboard electronics 126 and control of various functions provided by the onboard electronics 126, including camera functions, display functions, and optical shielding functions. In this example, the input mechanism includes a pair of push buttons 130 mounted on the frame 112 so as to be accessible over the respective ends 118 for the user to press. In other embodiments, additional and / or alternative user input mechanisms may be provided by, for example, one or more touch pads for receiving tactile input, such as located on one or both temples 114. Alternatively or additionally, some examples provide control of one or more device functions by a tap sequence manually applied to the body 108 of the AR glasses 100.

[0077] One of the buttons 130 in this example allows control of photo and video capture via camera 128. Another button 130 in this example serves as a manual input device for a selective control mechanism that provides user-controlled adjustment and / or switching of the optical mode of lens assembly 110 via shielding mechanism 102.

[0078] Now go to Figure 2A, it can be seen that in this example, the shielding mechanism 102 is configured for powered changes in opacity. In particular, the example AR glasses 100 include an actuator in the example form of a stepper motor 208, which is housed in the top rod of the frame 112 and is engaged with an annular frame ring 206, which is fixed to the radial periphery of the polarizing filter 124 and is rotatably mounted within the associated lens frame 116 and the top rod of the frame 112. Therefore, activation of the stepper motor 208 causes powered rotation of the frame ring 206, and therefore causes powered rotation of the polarizing filter 124 about its rotational axis 120. In this example, the stepper motor 208 and the frame ring 206 are engaged via a meshing gear structure on the outer diameter of the frame ring 206, but it should be understood that any suitable mechanical drive mechanism may be used in other examples.

[0079] In this example, the button 130 for opacity control is operably connected to the stepper motor 208 via the opacity controller 204, which is provided by one or more processors forming part of the onboard electronics 126. In some examples, the operation of the opacity control button 130 is bimodal, where a single button press switches the shielding mechanism 102 between a maximum transmissive state and a maximum opaque state. However, in this example, the user can select any opacity level between these limits by means of a press and hold input, in response to which the polarizing filter 124 is smoothly rotated, stopping at the desired angular position in response to the release of the button 130. In this way, the user can optionally darken the lens assembly 110 to adjust the ambient brightness without making them functionally opaque to allow the AR glasses 100 to be used in the manner of sunglasses.

[0080] In the present example, the opacity controller 204 is additionally configured to provide dynamic autonomous mode switching through operation of the stepper motor 208. To this end, the opacity controller 204 continuously monitors sensor and usage data and, in response to identifying that predefined criteria are satisfied, autonomously activates the stepper motor 208 to switch the shielding mechanism 102 from its see-through state to its opaque state, or vice versa. Figure 6 and Figure 7 Further details regarding such autonomous opacity control are discussed.

[0081] In this example, the display mechanism 106 is provided by a near-eye display 210 integrated with the lens assembly 110. In this example, the near-eye display 210 includes a pair of laterally symmetrical forward optical assemblies housed in respective ends 118, each of which includes a projector 212 coupled to a fixed lens 122 of an associated lens assembly 110 (see FIG. FIG. 2A to FIG. 3B) to present a visual image in the corresponding display area 214. Therefore, each lens assembly 110 has a dedicated projector 212 serving the corresponding display area 214, as can be seen in FIG. Figure 3A Best seen in.

[0082] In this example, the near-eye display is provided with at least a portion of a fixed lens 122 that acts as a waveguide combiner. To this end, each fixed lens waveguide has an input region and an output region provided by a diffractive surface relief grating, wherein a display region 214 is defined by the corresponding surface relief grating. The waveguide includes a reflective structure or a diffractive structure (e.g., a grating and / or an optical element such as a mirror, lens or prism). Projected light 216 from the projector 212 thus enters the waveguide provided by the fixed lens 122 at an input region adjacent to the periphery of the lens assembly 110, traveling in a waveguide manner via the fixed lens 122 to encounter the diffractive structure defining the display region 214. The diffractive structure defining the display region 214 directs the projected light 216 toward the corresponding eye of the wearer, thereby providing an image on or in the lens assembly 110 that is superimposed on a view of the real world seen by the user. It should be understood that the display so provided in the display region 214 is transparent, so that the display is seen as superimposed on the visible environment.

[0083] However, it should be understood that other display technologies or configurations capable of displaying images to a user in a forward field of view may be provided. For example, instead of projector 212 and waveguide, an LCD, LED or other display panel or surface may be provided instead.

[0084] exist Figure 2A , the shielding mechanism 102 is in its perspective state (corresponding to the following Figure 3A ), wherein the polarization filter 124 on the fixed lens 122 and the corresponding polarization axes of the polarizing film 202 are aligned, thereby allowing the AR glasses 100 to operate in the AR mode. Figure 2A As shown in the schematic cross-sectional side view of , ambient light 218 passes through the lens assembly 110 in this state and reaches the wearer's eyes. Combined with the projected light 216 from the display area 214 of the near-eye display, the view of the real world environment provided by the ambient light 218 allows an AR experience to be provided by the near-eye display.

[0085] In contrast, Figure 2B (corresponding to Figure 3B) shows the shielding mechanism 102 in an opaque state, wherein the AR glasses 100 operate in VR mode or cinema mode as previously discussed. In this state, the polarization axes of the polarizing filter 124 and the polarizing film 202 are perpendicular to each other, effectively blocking any ambient light 218 from passing therethrough. Thus, the ambient light 218 passes through the polarizing filter 124, receiving the corresponding polarization. However, upon impinging on the static polarizer provided by the polarizing film 202, the polarized light is blocked from passing further. Thus, the projected light 216 delivered to the user via the display area 214 is visible to the wearer against an opaque, blackened visual background.

[0086] It will be seen that the objective intensity of the near-eye display 210 is unaffected by the optical state of the shielding mechanism 102 because, from the wearer's perspective, the waveguide provided by the fixed lens 122 and the diffractive structure defining the display area 214 are completely located in front of the polarizer of the shielding mechanism 102.

[0087] Figure 3A and Figure 3B The AR glasses 100 are shown from the wearer's perspective during different operating modes in which the lens assembly 110 is set to different optical conditions by operation of the shielding mechanism 102 provided by a pair of stacked polarizers, which in this example include a polarizing filter 124 and a polarizing film 202. For clarity, the optical properties of the lens assembly 110 have been omitted. Figures 1 to 2B As shown in the multiple components. Figure 1 As described in , the lens assembly 110 of the AR glasses 100 includes a pair of laterally spaced composite lens assemblies 110 fixed within corresponding lens rims 116 provided by a frame 112.

[0088] In these views, a fixed polarization axis 302 provided by polarizing film 202 on fixed lens 122 and an adjustable polarization axis 304 provided by polarizing filter 124 are used for increased visibility shown beneath respective lens assemblies 110 , although it should be understood that these axes coincide with lens assemblies 110 .

[0089] In particular, Figure 3A The view shows the AR glasses 100 in the AR mode (corresponding to Figure 2A ), during which the lens assembly 110 is in a transparent or translucent see-through state. In this example, the fixed polarization axis 302 and the adjustable polarization axis 304 are parallel and do not filter out more ambient light 218 than a single polarizer. Therefore, the viewer has a substantially clear view of their real-world environment 306.

[0090] Thus, the near-eye display 210 is able to present AR objects 308 that appear to be superimposed on the real-world environment 306. In this example, the AR objects 308 are presented in the left and right display areas 214 in a stereoscopic effect, such that they appear to be three-dimensional and located at a depth from the surrounding real-world environment 306. The visual content that can be displayed by the near-eye display 210 in AR mode is not limited to stereoscopic and / or superimposed AR content, but also includes conventional 2D material, such as graphical user interfaces, photos, and video content, such as movies, television, sporting events, and online video clips or social media stories.

[0091] In contrast, Figure 3B The AR glasses 100 are shown in a VR mode or a cinema mode, during which the lens assembly 110 is in an opaque state and visual content is presented in the display area 214 via the near-eye display 210. In this example, the visual content displayed by the near-eye display 210 is conventional 2D video content 310 in the form of a movie. It should be understood that when the shielding mechanism 102 is also in the optically light-blocking state, the dark mode or sleep mode of the AR glasses 100 is in operation, and the display mechanism 106 does not present anything.

[0092] In this example, the lens assembly 110 and the shielding mechanism 102 are completely opaque, substantially completely blocking ambient light from passing therethrough. Figure 3B , the adjustable polarization axis 304 and the fixed polarization axis 302 are orthogonal or at right angles to each other. In other embodiments, the maximum opacity of the lens assembly 110 allows at least some light to pass therethrough, but is insufficient to provide the wearer with a functionally clear view of their surroundings, and is therefore functionally opaque or non-transparent. It should also be understood that, such as Figure 3B The VR mode or theater mode in may optionally be provided with a shielding mechanism 102 that blocks most but not all ambient light 218 at the discretion of the user.

[0093] Likewise, where opacity levels between maximum transparency and maximum opacity can be selectively achieved, as is the case in this example, Figure 3A The AR mode can operate with the ambient light 218 slightly dimmed at the user's discretion. This can be used to dim overly bright ambient conditions, or to increase the perceived brightness of the near-eye display 210. Note that because (e.g. Figure 2BThe shielding mechanism 102 blocks ambient light 218 behind the waveguide and display area 214 provided by the fixed lens 122 (as viewed from the wearer's perspective), so dimming of the ambient light 218 through the optical shielding does not dim the objective intensity of the near-eye display 210. For this reason, any dimming of the ambient light 218 via a pair of polarizers increases the subjectively perceived intensity and, therefore, the clarity of the display. Thus, in some cases, the wearer can adjust or modulate display clarity by selectively changing the lens opacity through operation of the shielding mechanism 102.

[0094] When the shielding mechanism 102 is in its fully opaque state ( Figure 3B ), the visual lens assembly 110 to the near eye display 210 is completely shielded or blocked. This allows a VR experience to be provided via the near eye display 210. Conversely, when the real world surroundings are visible in the background, the wearer can choose to view 2D non-AR visual content via the near eye display 210 without any visual distractions in the visual lens assembly 110, as is the case in AR mode ( Figure 3A ).

[0095] System with glasses supporting electronic devices

[0096] Figure 4 A system is shown in which glasses supporting electronic devices, such as example AR glasses 100, can be implemented according to one example. Figure 4 is a high-level functional block diagram of an example pair of AR glasses 100 communicatively coupling a mobile client device 402 and a server system 424 via various networks 430 .

[0097] As briefly discussed previously, the AR glasses 100 include a camera device 128 , a near-eye display 210 , an optical shielding mechanism 102 , and an opacity controller 204 for autonomous control of the shielding mechanism 102 .

[0098] The client device 402 may be a smartphone, tablet, phablet, laptop, wireless access point, or any other such device capable of connecting to the AR glasses 100 using either or both of a low-power wireless connection 426 and a high-speed wireless connection 428. The client device 402 is connected to a server system 424 and a network 430. The network 430 may include any combination of wired and wireless connections.

[0099] The AR glasses 100 also include two image displays of the near-eye display 210. The two image displays include one image display associated with the left lateral side of the AR glasses 100 and one image display associated with the right lateral side of the AR glasses 100. The AR glasses 100 also include an image display driver 406, an image processor 408, a low-power circuit system 418, and a high-speed circuit system 410. The near-eye display 210 is configured to present images and videos to a user of the AR glasses 100, including images that may include a graphical user interface.

[0100] The image display driver 406 commands and controls the image display of the near-eye display 210. The image display driver 406 may transmit the image data directly to the image display of the near-eye display 210 for presentation, or may have to convert the image data into a signal or data format suitable for transmission to the image display device. For example, the image data may be video data formatted according to a compression format such as H.264 (MPEG-4 Part 10), HEVC, Theora, Dirac, RealVideo RV40, VP8, VP9, ​​etc., and the still image data may be formatted according to a compression format such as Portable Network Graphics Group (PNG), Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), or Exchangeable Image File Format (Exif).

[0101] As described above, the AR glasses 100 include a frame 112 and a stem (or temple) extending from a lateral side of the frame 112. The AR glasses 100 also include one or more user input devices 404, which in this example include a touch sensor and a push button 130. The user input device 404 (e.g., a touch sensor or a push button) is used to receive input selections from a user to manipulate a graphical user interface of a presented image.

[0102] Figure 4 The components of the AR glasses 100 shown in the figure are located on one or more circuit boards (e.g., PCBs or flexible PCBs) in the frame or temples. Alternatively or additionally, the depicted components may be located in a block, frame, hinge, or beam of the AR glasses 100. The left camera and the right camera 128 may include digital camera elements, such as complementary metal oxide semiconductor (CMOS) image sensors, charge coupled devices, or any other corresponding visible light or light capturing elements that can be used to capture data including images of scenes with unknown objects.

[0103] The AR glasses 100 include a memory 414 that stores instructions for performing a subset or all of the functions described herein. The memory 414 may also include a storage device. In this example, the memory 414 stores instructions for dynamic autonomous mode switching, thereby implementing the opacity controller 204 by executing instructions through the high-speed circuit system 410.

[0104] like Figure 4 As shown in , the high-speed circuit system 410 includes a high-speed processor 412, a memory 414, and a high-speed wireless circuit system 416. In this example, the image display driver 406 is coupled to the high-speed circuit system 410 and operated by the high-speed processor 412 to drive the left image display and the right image display of the near-eye display 210. The high-speed processor 412 can be any processor capable of managing high-speed communications and operations of any general-purpose computing system required for the AR glasses 100. The high-speed processor 412 includes the processing resources required to manage high-speed data transmission on a high-speed wireless connection 428 to a wireless local area network (WLAN) using the high-speed wireless circuit system 416. In some examples, the high-speed processor 412 executes an operating system, such as a LINUX operating system or other such operating system of the AR glasses 100, and the operating system is stored in the memory 414 for execution. In addition to any other duties, the high-speed processor 412, which executes the software architecture of the AR glasses 100, is used to manage data transmission with the high-speed wireless circuit system 416. In some examples, high-speed wireless circuit system 416 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 902.11 communication standard, also referred to herein as Wi-Fi. In other examples, other high-speed communication standards can be implemented by high-speed wireless circuit system 416.

[0105] The low-power wireless circuit system 422 and the high-speed wireless circuit system 416 of the AR glasses 100 may include a short-range transceiver (Bluetooth TM ) and a wireless wide area network transceiver, a local area network transceiver, or a wide area network transceiver (e.g., cellular or WiFi). The client device 402 including a transceiver that communicates via a low power wireless connection 426 and a high speed wireless connection 428 can be implemented using details of the architecture of the AR glasses 100, as can other elements of the network 430.

[0106] The memory 414 includes any storage device capable of storing various data and applications, including camera data generated by the left and right cameras 128, the opacity controller 204, and the image processor 408, and images generated by the image display driver 406 for display on the image display of the near-eye display 210, etc. Although the memory 414 is shown as being integrated with the high-speed circuit system 410, in other examples, the memory 414 can be a separate independent element of the AR glasses 100. In some such examples, electrical wiring can provide a connection from the image processor 408 or the low-power processor 420 to the memory 414 through a chip including the high-speed processor 412. In other examples, the high-speed processor 412 can manage the addressing of the memory 414 so that the low-power processor 420 will start the high-speed processor 412 any time a read or write operation involving the memory 414 is required.

[0107] like Figure 4 As shown in , the low-power processor 420 or the high-speed processor 412 of the AR glasses 100 can be coupled to the camera 128, the shielding mechanism 102 (in this example, the shielding mechanism 102 includes the stepper motor 208, the low-power processor 420 and / or the high-speed processor 412, thereby at least partially providing the opacity controller 204), the image display driver 406, the user input device 404 (for example, a touch sensor or a push button), and the memory 414.

[0108] The AR glasses 100 are connected to a host computer. For example, the AR glasses 100 are paired with the client device 402 via a high-speed wireless connection 428, or connected to a server system 424 via a network 430. The server system 424 can be one or more computing devices as part of a service or network computing system, for example, the one or more computing devices include a processor, a memory, and a network communication interface that communicates with the client device 402 and the AR glasses 100 through the network 430.

[0109] The client device 402 includes a processor and a network communication interface coupled to the processor. The network communication interface allows communication via a network 430, a low-power wireless connection 426, or a high-speed wireless connection 428. The client device 402 may also store at least a portion of the instructions for autonomous and / or user-directed opacity control functionality performed via the optical shielding mechanism 102 described herein. Thus, in some examples, opacity control may be performed entirely on the device at the AR glasses 100. In other embodiments, at least a portion of the opacity controller 204 may be provided by the connected client device 402. In one such embodiment, the client device 402 has installed thereon a machine learning model (ML model) derived from a neural network, which is trained using use case training data related to sensor data and use case data, where optical shielding is appropriate or manually / selectively implemented by a user. In such a case, the ML model autonomously triggers the shielding mechanism 102, and therefore the lens assembly 110, to switch between different optical modes, as described herein and as described below with reference Figure 7 In some embodiments, such an ML model can be implemented by the on-device opacity controller 204. In other embodiments, autonomous opacity control can be provided at least in part by the server system 424, for example, by using a continuously updated AI system implemented by the server system 424 to control multiple AR glasses 100 communicating with it via the network 430.

[0110] The output components of the AR glasses 100 include visual components, such as a near-eye display 210 and / or a light-emitting diode (LED) display. The image display of the near-eye display 210 is driven by the image display driver 406. The output components of the AR glasses 100 also include acoustic components (e.g., speakers), tactile components (e.g., vibration motors), other signal generators, etc. The input components of the AR glasses 100, the client device 402, and the server system 424, such as the user input device 404, may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric inputs, optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing tools), tactile input components (e.g., buttons 130, touch screens or other tactile input components that provide touch positions and forces or touch gestures), audio input components (e.g., microphones), etc.

[0111] The AR glasses 100 may optionally include additional peripheral device elements. Such peripheral device elements may include onboard device sensors 432, including biometric sensors, motion sensors, and positioning sensors integrated with the AR glasses 100 in this example. For example, the peripheral device elements may include any I / O components including output components, motion components, position components, or any other such components described herein.

[0112] For example, the biometric components of the device sensor 432 include components for the following operations: detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biological signals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion components of the device sensor 432 include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. The positioning components include position sensor components for generating position coordinates (e.g., global positioning system (GPS) receiver components), WiFi or Bluetooth for generating positioning system coordinates. TM transceiver, altitude sensor components (e.g., an altimeter or barometer that detects air pressure, from which altitude can be derived), orientation sensor components (e.g., a magnetometer), etc. Such positioning system coordinates can also be received from the client device 402 via the low-power wireless circuit system 422 or the high-speed wireless circuit system 416 through the low-power wireless connection 426 and the high-speed wireless connection 428.

[0113] When phrases similar to "at least one of A, B, or C," "at least one of A, B and C," "one or more A, B, or C," or "one or more of A, B, and C" are used, it is intended that the phrase be interpreted to mean that A can exist alone in an embodiment, B can exist alone in an embodiment, C can exist alone in an embodiment, or any combination of elements A, B, and C can exist in a single embodiment; for example, A and B, A and C, B and C, or A and B and C.

[0114] Changes and modifications may be made to the disclosed embodiments without departing from the scope of the present disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure as expressed in the appended claims.

[0115] Networked computing environment

[0116] Figure 5 is a block diagram illustrating an example messaging system 500 for exchanging data (eg, messages and associated content) over a network. The messaging system 500 includes a system similar to that described with reference to FIG. Figure 4 Multiple instances of the illustrated client device 402, each instance hosting a number of applications, including a messaging client 502 and other applications 504. In this example, the client device 402 is coupled to a client similar to the one described in reference Figures 1 to 4 A mobile phone is depicted with a pair of AR glasses 100. In other examples, the client device 402 may be provided by the AR glasses 100 themselves.

[0117] Each messaging client 502 is communicatively coupled to other instances of messaging clients 502 (e.g., hosted on respective other client devices 402), messaging server systems 506, and third-party servers 508 via a network 510 (e.g., the Internet). The messaging client 502 may also communicate with locally hosted applications 504 using an application programming interface (API). In some examples, the messaging client 502 may be implemented by a server such as a server. Figure 1 Wearable devices such as AR glasses 100 Figure 4 The client device 402 or a combination thereof is provided.

[0118] The messaging clients 502 are able to communicate and exchange data with other messaging clients 502 and with a messaging server system 506 via a network 510. The data exchanged between messaging clients 502 and between messaging clients 502 and messaging server system 506 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0119] The messaging server system 506 provides server-side functionality to a particular messaging client 502 via the network 510. Although certain functionality of the messaging system 500 is described herein as being performed by the messaging client 502 or by the messaging server system 506, it may be a design choice whether certain functionality is located within the messaging client 502 or within the messaging server system 506. For example, it may be technically preferable to initially deploy certain technologies and functionality within the messaging server system 506, but later migrate the technologies and functionality to the messaging client 502, where the client device 402 has sufficient processing power.

[0120] The messaging server system 506 supports various services and operations provided to the messaging client 502. Such operations include sending data to the messaging client 502, receiving data from the messaging client 502, and processing data generated by the messaging client 502. As an example, the data may include message content, client device information, geographic location information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. The data exchange within the messaging system 500 is stimulated and controlled by functions available via the user interface (UI) of the messaging client 502.

[0121] Turning now specifically to the messaging server system 506, an application program interface (API) server 514 is coupled to and provides a programming interface to the application server 512. The application server 512 is communicatively coupled to a database server 518, which facilitates access to a database 524 that stores data associated with messages processed by the application server 512. Similarly, a web server 526 is coupled to and provides a web-based interface to the application server 512. To this end, the web server 526 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0122] The application program interface (API) server 514 receives and sends message data (e.g., commands and message payloads) between the client device 402 and the application server 512. Specifically, the application program interface (API) server 514 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client 502 to activate the functions of the application server 512. The application program interface (API) server 514 exposes various functions supported by the application server 512, including: account registration; login functionality; sending messages from a particular messaging client 502 to another messaging client 502 via the application server 512; sending media files (e.g., images or videos) from the messaging client 502 to the messaging server 516 and for possible access by another messaging client 502; setting of media data collections (e.g., stories); retrieving a friend list of the user of the client device 402; retrieving such collections; retrieving messages and content; adding and removing entities (e.g., friends) from an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to the messaging client 502).

[0123] The application server 512 hosts several server applications and subsystems, including, for example, a messaging server 516, an image processing server 520, and a social network server 522. The messaging server 516 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 502. As will be described in more detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to the messaging client 502. In view of the hardware requirements for other processor and memory intensive data processing, other processor and memory intensive data processing can also be performed on the server side by the messaging server 516.

[0124] The application server 512 also includes an image processing server 520 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 516.

[0125] The social network server 522 supports various social networking functions and services and makes them available to the messaging server 516. To this end, the social network server 522 maintains and accesses an entity graph within the database 524. Examples of functions and services supported by the social network server 522 include identifying other users in the messaging system 500 who have a relationship with a particular user or who the particular user is "following," and also identifying interests and other entities of a particular user.

[0126] Returning to the messaging client 502, the features and functions of the external resource (e.g., application 504 or applet) are available to the user via the interface of the messaging client 502. In this context, "external" refers to the fact that the application 504 or applet is outside the messaging client 502. External resources are usually provided by a third party, but can also be provided by the creator or provider of the messaging client 502. The messaging client 502 receives a user selection of an option to launch or access the features of such an external resource. The external resource can be an application 504 (e.g., a "local app") installed on the client device 402, or a small-scale version of the application (e.g., a "applet") hosted on the client device 402 or away from the client device 402 (e.g., on a third-party server 508). The small-scale version of the application includes a subset of the features and functions of the application (e.g., a full-scale, local version of the application) and is implemented using a markup language document. In one example, the small-scale version of the application (e.g., a "applet") is a web-based markup language version of the application and is embedded in the messaging client 502. In addition to using markup language documents (eg, *ml files), applets may include scripting languages ​​(eg, .*js files or .json files) and style sheets (eg, *ss files).

[0127] In response to receiving a user selection of an option to launch or access a feature of an external resource, the messaging client 502 determines whether the selected external resource is a web-based external resource or a locally installed application 504. In some cases, the application 504 locally installed on the client device 402 can be independent of the messaging client 502 and launched separately from the messaging client 502, such as by selecting an icon corresponding to the application 504 on a home screen of the client device 402. A small-scale version of such an application can be launched or accessed via the messaging client 502, and in some examples, all portions of the small-scale application cannot be accessed outside of the messaging client 502 or limited portions of the small-scale application can be accessed outside of the messaging client 502. The small-scale application can be launched by the messaging client 502, for example, receiving a markup language document associated with the small-scale application from a third-party server 508 and processing such a document.

[0128] In response to determining that the external resource is a locally installed application 504, the messaging client 502 instructs the client device 402 to launch the external resource by executing the locally stored code corresponding to the external resource. In response to determining that the external resource is a web-based resource, the messaging client 502 communicates with the third-party server 508 (for example) to obtain a markup language document corresponding to the selected external resource. The messaging client 502 then processes the obtained markup language document to present the web-based external resource within the user interface of the messaging client 502.

[0129] The messaging client 502 can notify the user of the client device 402 or other users (e.g., "friends") related to such a user of one or more external resources of activities that are occurring. For example, the messaging client 502 can provide a notification related to the current or recent use of an external resource by one or more members in a user group to a participant in a conversation (e.g., a chat session) in the messaging client 502. One or more users can be invited to join a valid external resource or to start an external resource that was recently used but is currently inactive (in the friend group). The external resource can provide the participants in the conversation who each use the corresponding messaging client 502 with the ability to share items, conditions, states, or locations in the external resource with one or more members in the user group entering the chat session. The shared item can be an interactive chat card, and the members of the chat can interact with the interactive chat card, for example, to start the corresponding external resource, view specific information in the external resource, or bring the members of the chat to a specific location or state in the external resource. In a given external resource, a response message can be sent to the user on the messaging client 502. The external resource may selectively include different media items in the response based on the current context of the external resource.

[0130] Messaging client 502 may present a list of available external resources (e.g., applications 504 or applets) to the user to launch or access a given external resource. The list may be presented in the form of a context-sensitive menu. For example, icons representing different applications (or applets) of application 504 (or applets) may vary based on how the user launches the menu (e.g., from a conversational interface or from a non-conversational interface).

[0131] Example Method

[0132] Figure 6 is a high-level flow chart of a method 600 for autonomously controlling an optical state or operating mode of an eyewear device having an optical shielding mechanism and supporting an electronic device according to one example. Figures 1 to 4 The method is further described using the example AR glasses 100, but it should be understood that the described technology can be implemented with different example devices in other situations.

[0133] Figure 6 In operation 602, operation data from an eyewear device (e.g., AR glasses 100) having an optical system (e.g., lens assembly 110) capable of switching between different optical states is accessed, wherein the different optical states include at least (a) a see-through state (e.g., Figure 2A and Figure 3A ) and (b) opaque states (e.g. Figure 2B and Figure 3B ), in which opaque state the wearer's view of their environment is obscured by the lens assembly. The operational data includes a current optical state of the optical system and sensor data captured by one or more sensors integrated into the eyewear device (e.g., device sensor 432). Method 600 also includes: at operation 604, when the lens assembly is in a first optical state of its optical states (e.g., see-through), identifying satisfaction of a predefined mode switching criterion applicable to the first optical state (e.g., the wearer is lying down and has fallen asleep or their eyes have closed). At operation 606, in response to satisfying the applicable mode switching criterion, the method includes automatically implementing setting of the optical system from the first optical state of the lens assembly's optical state to a different second optical state (e.g., switching the lens assembly 110 to fully opaque for operation of the device as a sleep mask).

[0134] At operation 710, now go to Figure 7 , flowchart 700 schematically illustrates a more detailed method of operating an eyewear device according to some examples. For ease of reference, Figure 7 The method will be further described as being composed and passing Figures 1 to 4 The example shown is performed by the AR glasses 100 consistent with the example shown, but it should be understood that in other examples, glasses devices with different configurations can be used to perform similar methods. In addition, in this example, the operations of flowchart 700 are performed by the onboard electronics 126 of the AR glasses 100, and specifically by the opacity controller 204, but in other embodiments, they can be at least partially performed by the coupled client device 402 ( Figure 4 ) and / or server system 424. In some examples, a machine learning model (ML model) can be implemented at least in part on the server side, such as by server system 424, through which at least some autonomous decision-making operations can be performed in some operations.

[0135] In operation 702, the opacity controller 204 (or in other examples, off-board decision circuitry) accesses current or real-time operational data of the AR glasses 100. In this example, such operational data includes:

[0136] a. Usage data from the glasses 100 (e.g., whether any visual content is being displayed using the near-eye display 210, and if so, the type of visual content currently being displayed; the current optical state or mode of the lens system 104, e.g., opaque or at least partially transparent; etc.);

[0137] as well as

[0138] b. Sensor data captured by integrated onboard device sensors 432 (e.g., motion data, biometric data, processed data from a camera facing where to determine whether the wearer's eyes are open or closed, geolocation data, etc.).

[0139] At operation 702, the current operating data is processed to determine the current operating mode of the AR glasses 100. In this example, the AR glasses 100 are bimodal and can operate in an AR mode (in the AR mode, the shielding mechanism makes the lens assembly transparent or at least semi-transparent; see, for example, Figure 2A or Figure 3A ), or capable of operating in VR mode (in VR mode, the polarization axes 302, 304 of the polarizing filter 124 and the fixed lens 122 are more or less perpendicular to each other, making the lens assembly 110 functionally opaque; see, e.g. Figure 2B or Figure 3B ).

[0140] If it is determined at operation 702 that the AR glasses 100 are currently in AR mode, then at operation 712, the opacity controller continuously monitors operational data from the AR glasses 100 to identify satisfaction of criteria for switching the AR glasses 100 from AR mode to VR mode (for ease of reference, further referred to as AR-VR switching). On the other hand, if it is determined at operation 702 that the glasses are currently in VR mode, then at operation 714, the opacity controller continuously monitors operational data to identify satisfaction of criteria for switching the AR glasses 100 from VR mode to AR mode (for ease of reference, further referred to as VR-AR switching). Note that the applicable mode switching criteria for AR-VR switching are different from the applicable mode switching criteria for VR-AR switching. This is because the conditions under which vision is autonomously obstructed are generally different from the conditions under which unobstructed vision is desirable or necessary.

[0141] In this example, a plurality of predefined different sets of AR-VR switching criteria are provided (all of which are to be satisfied). In particular, the AR-VR switching is configured in this example to automatically switch the lens system from its see-through state ( Figure 3A ) to its opaque state ( Figure 3B ),Right now:

[0142] a. Using the glasses' integrated near-eye display to view non-AR visual content (e.g., watching movies, sporting events, etc. in Cinema Mode; or in VR Mode to provide a VR experience) while the wearer is stationary or non-walking (i.e., sitting or lying / lying down); and

[0143] b. Using the AR glasses 100 as a sleeping mask or protective cover, where the shielding mechanism 102 makes the lens system 104 completely opaque without any visual content being reproduced by the near-eye display 210.

[0144] In some examples, the mode switching criteria are explicitly defined in each case as a set of corresponding variables or parameters with associated thresholds, each of which is required to be satisfied to achieve autonomous mode switching. Alternatively or in combination, in some examples autonomous mode switching is implemented using a machine learning model (ML model), through the operation of which the satisfaction of at least some mode switching criteria is identified based on current operating data. For example, the identification of the wearer's activity state can be determined nonlinearly by the ML model in some cases, while other combined criteria (such as display activity) can be determined by the opacity controller 204, and these facts are combined to determine the autonomous opacity control performed via the shielding mechanism 102. Examples of such ML model-determined behavioral states include whether the wearer has fallen asleep or intends to fall asleep, whether he has gone to bed, whether he has gotten up after resting, whether he is sitting or otherwise stationary in a traffic environment while viewing non-AR visual content, etc.

[0145] For example, in one example, the standard for mode switching to sleep mode states:

[0146] i. The wearer is lying down (e.g.,

[0147] prone or lying down);

[0148] ii. The wearer is resting (as indicated by motion data below a threshold and / or by biometric sensing

[0149] data); and

[0150] iii. The wearer has not actively chosen to display visual content via the integrated near-eye display 210 .

[0151] Similarly, in this example, at operation 714, the criteria for VR-AR switching include:

[0152] i. The AR glasses 100 are currently in sleep mode. When the near-eye display 210 is not in use, the

[0153] The mirror assembly 110 is completely shielded by the shielding mechanism 102;

[0154] ii. Biometric sensor data from AR glasses 100 based on predefined parameter thresholds or

[0155] ML model evaluation indicates that the wearer is waking up; and

[0156] iii. The motion data from the AR glasses 100 is analyzed according to predefined parameter thresholds or ML models

[0157] The assessment indicates that the user is getting out of bed;

[0158] ·or

[0159] i. The AR glasses 100 are currently in cinema mode, when the wearer is sitting and viewing the near-eye display

[0160] When the device 210 is actively watching a movie, the lens assembly 110 is currently blocked;

[0161] ii. the film is about to end; and

[0162] iii.The wearer is getting up.

[0163] ·or

[0164] i. The AR glasses 100 are currently in cinema mode, and the lens assembly 110 is currently blocked when the wearer is actively watching a movie via the near-eye display 210;

[0165] ii. The wearer is stationary in the rail transit vehicle; and

[0166] iii. The vehicle arrives at the wearer's destination.

[0167] It should be understood that the examples provided above are not exhaustive, and various different mode switching criteria may be employed in other examples. Therefore, if the applicable mode switching criteria are met at determination operation 708, then at operation 716, the shielding mechanism 102 of the AR glasses 100 is switched to the opposite optical state, that is, the shielding mechanism 102 is switched from an opaque state to a see-through state, or vice versa, depending on the currently active optical mode.

[0168] After the mode switch, at operation 716, the opacity controller 204 resumes monitoring of operational data from the AR glasses 100 at operation 712 or operation 714, as the case may be, to identify satisfaction of newly applicable mode switch criteria that are specific to the particular optical mode to which the AR glasses were most recently switched. Note that in this example, the wearer may provide a manual or selective command to switch the optical state of the lens system 104 at any time. In this example embodiment, a single press of the left push button 130 causes the shielding mechanism 102 to change the optical state of the lens assembly 110 from its current optical mode to the opposite optical mode. Thus, when the lens assembly 110 is obscured, a single press on the left push button 130 causes an actuated rotation of the polarizing filter 124 to switch the lens assembly 110 to a see-through state, and vice versa. In other examples, the mode switch command may alternatively or additionally be provided via a linked mobile device, such as Figure 4 The client device 402 provides.

[0169] In this example, the pressing and holding operation of the left push button 130 implements selective graduated opacity control by causing continuous actuation rotation of the polarizing filter 124 until the button 130 is released. In this way, the user can adjust the opacity of the lens assembly 110 to suit their personal situational preferences.

[0170] Note that in all of the above examples, the shielding mechanisms 102 of the left and right lens assemblies 110 are linked by their corresponding stepper motors 208 for synchronized opacity balancing, so that the opacity values ​​of the left and right lens assemblies are always kept equal to each other. FIG. 8A to FIG. 8C Those examples are described in which the rotational mechanisms of polarizing filters 124 are mechanically linked such that manual or actuated adjustment of the rotational position of one of polarizing filters 124 is automatically reflected by the other polarizing filter 124 .

[0171] In addition, some examples of both AR glasses and non-AR smart glasses provide dynamic autonomous opacity adjustment or modulation in response to ambient brightness levels while maintaining a see-through or translucent state. In this way, the shielding mechanism 102 effectively provides a dynamically automatically modulated sunglass-style tint level of the lens assembly 110, thereby facilitating the multifunctional use of the AR glasses 100. The AR glasses 100 include an example of such a tint modulation system, which is composed of an opacity controller 204 that dynamically adjusts the rotational position of the polarizing filter 124 to adjust the composite opacity level or tint level of the lens system 104.

[0172] At operation 704, when the AR glasses 100 are in AR mode, the opacity controller 204 continuously determines the ambient brightness value experienced by the AR glasses 100 based on the operational data received at operation 710. In this example, the ambient brightness is indicated by the current or most recent reading of a light meter contained in the glasses body. In some examples where an inherent light meter is not provided, the ambient brightness level is calculated by processing contemporaneous image data captured by the camera 128.

[0173] If the user has selectively enabled active tint modulation (e.g., via a linked mobile phone application), the opacity controller 204, when in operation, automatically modulates the lens tint level by dynamically and automatically adjusting (if necessary) the opacity level of the shielding mechanism by driving the polarizing filter to rotate to a corresponding angular position relative to the rotation axis 120. In this example, the user can pre-select a target light value or brightness level to be maintained from the wearer's perspective, where the shielding mechanism 102 is in brighter conditions, and the target level is dynamically adjusted to filter out the amount of ambient light required to achieve the wearer's target light value.

[0174] It will be appreciated that the active tint modulation provided by the pair of polarizers of the shielding mechanism 102 is far superior to conventional techniques for providing variable tint sunglass lenses in terms of response over the available shielding range and response time (which is virtually instantaneous).

[0175] Note that in other examples, the above-described techniques for controlling the opacity (i.e., optical transmittance) of a head-mounted lens system can be used in non-AR devices, non-glasses devices, non-smart devices, and / or any feasible combination of these devices.

[0176] Thus, for example, one example provides a VR head-mounted device with an integrated shielding mechanism according to the disclosed examples (e.g., a head-mounted device configured to provide a fully immersive experience by fully utilizing a near-eye display to occupy the wearer's entire field of view). Although such devices are head-mounted and provide a near-eye display to the wearer, they are not configured or suitable for wear in outdoor and / or uncontrolled real-life environments, and are therefore not eyewear devices, which is consistent with the common meaning of the phrase used herein. The provision of an adjustable stacked polarizer shielding mechanism integrated into the lens system of the example VR head-mounted device provides the ability to quickly switch the head-mounted device from VR functionality to AR functionality by switching the lens system from an opaque state to a see-through state. Thus, when a VR user wishes to end their VR experience or urgently needs to view their environment, the lens system of the VR head-mounted device can be easily rendered functionally transparent, rather than physically removing the head-mounted device.

[0177] In some examples, such a variable-opacity head-mounted device provides AR functionality when its lens system is in a see-through mode, for example, providing an AR user interface for the head-mounted device and / or for one or more wirelessly linked devices. In such examples, the shielding mechanism is thus operable to switch the head-mounted device between AR mode and VR mode.

[0178] Additional examples include non-AR smart glasses (e.g., eyewear devices that support electronic devices without a near-eye display mechanism and are configured to present AR and / or VR content) that include in their lens systems a lens system that is similar to the reference image. Figures 1 to 7 In such an example, the shielding mechanism can be used as a sleep or rest mask under light-blocking optical conditions. Thus, when the shielding mechanism is in its fully opaque optical state, the non-AR smart glasses are in a sleep or rest mode and do not provide the same protection as in the case of a non-AR smart glasses. Figure 1 The option of cinema mode or VR mode in case of AR glasses.

[0179] In some examples of such non-AR smart glasses, autonomous mode switching functionality and / or autonomous dynamic tint level control is provided by the glasses' inherent onboard electronics, which is similar to or analogous to Figures 1 to 7 The opacity controller. In the containing and reference Figures 1 to 7 In other examples of smart glasses with shielding mechanisms similar to or similar to the shielding mechanisms described, the onboard electronics of the smart glasses do not provide autonomous opacity control. In such cases, opacity control of the lens system through operation of the shielding mechanism is exclusively provided directly by the user. In various examples, selective user control of the shielding mechanism includes:

[0180] a. In some examples, specifically via an electromechanical actuation mechanism (e.g., see Figure 2A Such stepper motor 208 is described, in this example controlled by a user pressing button 130);

[0181] b. In some examples, exclusively via a mechanical control mechanism (see, e.g., reference Fig. 8A To Figure

[0182] 8C Description of manually actuated synchronous rotation of movable polarizers); and

[0183] c. In other examples, alternatively through user operation of an integrated electromechanical control mechanism or an integrated mechanical control mechanism, depending on user preference.

[0184] at last, FIG. 8A to FIG. 8CAn example eyewear device in the form of a pair of glasses 800 that is not electronically supported is shown, which is referred to as a non-smart device consistent with the above terminology. Unless otherwise indicated, similar reference numerals refer to the same. Figures 1 to 3B similar parts in, and on the other hand refers to FIG. 8A to FIG. 8C Similar parts in .

[0185] The physical structure of the glasses 800 is similar to that of the glasses 800 described above. Figures 1 to 3B The AR glasses 100 described have a glasses body including a lens carrying frame 112 and a pair of hinged temples. The lens system through which the wearer's view of the environment is provided again includes laterally spaced apart lens assemblies 804.

[0186] Similar to Figure 2A In a physical optical arrangement best seen in FIG. 1 , each lens assembly 804 includes a pair of stacked polarizers provided by a polarizing fixed lens 122 (e.g., a composite lens including a transparent glass substrate structure with a polarizing film 202 in some examples, and a plate of polarizing glass or transparent polymer in other examples) and a polarizing filter 124 that is capable of rotating relative to the fixed lens 122 about a rotation axis 120 extending more or less along the wearer's line of sight, thereby together providing a shielding mechanism 102 to control the opacity of the lens assembly 804 by controlling the relative orientation of the respective polarization axes 302, 304 of the fixed lens 122 and the polarizing filter 124. In contrast to the lens assembly 110 of the AR glasses 100, the fixed lens 122 of the glasses 800 does not include or provide any display element or functionality.

[0187] Another difference between the non-intelligent glasses 800 and the AR glasses 100 in this example is that there is no electromechanical actuator in the non-intelligent glasses 800, for example, Figure 2A 8. In contrast, the angular positioning of the polarizing filter 124 (and therefore the misalignment angle between the polarization axes 302, 304, which in turn determines the composite opacity or transmittance of the lens assembly 804 to ambient light) can be mechanically adjusted by direct manual control of the user. In this example, the polarizing filter 124 of the right lens assembly 804 has a control piece 806 attached thereto that protrudes radially outward from the radial periphery of the lens frame within which the polarizing filter 124 is retained. This enables the user to change the rotational position of the polarizing filter 124 by pushing or nudging the control piece 806 circumferentially downward or upward.

[0188] In this example embodiment, the polarizing filters 124 of the left lens assembly 110 are connected by a mechanical linkage (e.g., by a gear system housed by the interior of the frame) to synchronize and mirror the movement of the laterally spaced pair of polarizing filters 124. Thus, for example, if the user changes the angular position of the right polarizing filter 124 by 20° through operation of the control piece 806, the left polarizing filter 124 automatically and immediately rotates an equal amount in synchronization with the initially moved right polarizing filter 124. In other examples, the respective lens assemblies 804 may be controlled individually.

[0189] Now turn to Fig. 8A , it will be seen that the glasses 800 are shown in a maximally transparent optical state, wherein the polarization axes 302, 304 of the stacked polarizers are aligned parallel to each other. Figure 8B In FIG. 1 , the user has moved the polarizing filter 124 from the center by rotating the control plate 806 clockwise from their viewing angle. Fig. 8A The position in the stacked polarizer is manually rotated approximately 45°. Therefore, the polarization axes 302, 304 of the stacked polarizers are at approximately 45° angles to each other, thereby increasing the opacity or tint of the lens assembly 804, dimming the perceived brightness of the surrounding environment according to the user's preference.

[0190] exist Figure 8C , the glasses 800 are shown in a sleep mode or rest mode, in which the shielding mechanism 102 is configured to place the lens system in a maximally opaque optical state. In this example, the polarizing filter 124 is able to be moved from its maximally transparent position ( Fig. 8A ) are shifted at least a full 90° so as to cause the polarization axes 302, 304 of the stacked polarizers to be at right angles to each other in a most opaque position to effectively block all ambient light.

[0191] The use case of the integrated shielding mechanism of lens assembly 804 for selectively varying the opacity or content of the lens system is similar to that described above with reference to the invention, except that modulation or adjustment of the opacity of the lens system is selectively performed manually by the user in all cases. Figures 1 to 7 The use cases described are similar or analogous (naturally excluding the functionality associated with the integrated near-eye display). Thus, indoors or under cloudy conditions, the wearer can easily place the polarizing filter 124 to Fig. 8A In outdoor or otherwise overly bright conditions, the wearer can manually adjust the tint or darkness of the glasses by rotating the control piece 806. In this manner, the glasses 800 provide adjustable sunglass functionality because a full range of tint levels that allow functional transparency under given conditions are available for the wearer to choose from.

[0192] However, unlike conventional sunglasses, the wearer has the ability to manually move the polarizing filter 124 to the Figure 8C The glasses 800 can be set to a maximum opaque position to completely block ambient light. In such an occluded state, the glasses 800 provide functionality similar to a sleep mask, allowing the user to relax, rest, or sleep more effectively in a bright environment. Thus, for example, when using public transportation or waiting to board a flight at an airport, the wearer can conveniently access substantially all of the functionality of a sleep mask by adjusting the glasses 800 to full darkness. In this way, a wearable device that the user already wears for other purposes is provided with additional sleep mask functionality, avoiding the need to carry and the hassles associated with a separate sleep mask, which is often lost and has only a single function.

[0193] Machine Architecture

[0194] Fig. 9The illustrative embodiment of the present invention is a diagrammatic representation of a machine 900 in which instructions 910 (e.g., software, programs, applications, applet, app, or other executable code) may be executed to cause the machine 900 to perform any one or more of the methods discussed herein. For example, the instructions 910 may cause the machine 900 to perform any one or more of the methods described herein. The instructions 910 convert a general purpose, unprogrammed machine 900 into a specific machine 900 that is programmed to perform the functions described and illustrated in the manner described. The machine 900 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a network deployment, the machine 900 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing instructions 910 specifying actions to be taken by the machine 900. In addition, although only a single machine 900 is shown, the term "machine" should also be taken to include a collection of machines that individually or jointly execute instructions 910 to perform any one or more of the methods discussed herein. For example, the machine 900 may include any of the client device 402 or a number of server devices that form part of the messaging server system 506. In some examples, the machine 900 may also include both a client system and a server system, wherein certain operations of a particular method or algorithm are performed on the server side, and certain operations of the particular method or algorithm are performed on the client side.

[0195] The machine 900 may include a processor 904, a memory 906, and an input / output I / O component 902, which may be configured to communicate with each other via a bus 940. In an example, the processor 904 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 908 and a processor 912 that execute instructions 910. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although Fig. 9 Multiple processors 904 are shown, but the machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0196] The memory 906 includes a main memory 914, a static memory 916, and a storage unit 918, which are all accessible by the processor 904 via the bus 940. The main memory 906, the static memory 916, and the storage unit 918 store instructions 910 that implement any one or more of the methods or functions described herein. The instructions 910 may also reside, completely or partially, within the main memory 914, within the static memory 916, within the machine-readable medium 920 within the storage unit 918, within at least one of the processors 904 (e.g., within a cache memory of a processor), or within any suitable combination thereof during execution thereof by the machine 900.

[0197] I / O components 902 may include various components that receive input, provide output, generate output, send information, exchange information, capture measurements, etc. The specific I / O components 902 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine will be less likely to include such a touch input device. It should be understood that I / O components 902 may include Fig. 9928. In various examples, the I / O component 902 may include a user output component 926 and a user input component 928. The user output component 926 may include a visual component (e.g., a near-eye display such as a plasma near-eye display panel (PDP), a light-emitting diode (LED) near-eye display, a liquid crystal near-eye display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 928 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a point-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or other pointing instrument), a tactile input component (e.g., a physical button, a touch screen or other tactile input component that provides the location and force of a touch or touch gesture), an audio input component (e.g., a microphone), etc.

[0198] In other examples, the I / O component 902 may include a biometric component 930, a motion component 932, an environmental component 934, or a position component 936, as well as a wide range of other components. For example, the biometric component 930 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 932 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).

[0199] Environmental components 934 include, for example, one or more cameras (with still image / photo and video capabilities), lighting sensor components (e.g., a photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., a barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., an infrared sensor that detects nearby objects), gas sensors (e.g., a gas detection sensor for detecting concentrations of hazardous gases for safety purposes or for measuring pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0200] With respect to cameras, client device 402 can have a camera system including, for example, a front-facing camera on a front surface of client device 402 and a rear-facing camera on a rear surface of client device 402. The front-facing camera can, for example, be used to capture still images and videos of a user of client device 402 (e.g., a "selfie"), which can then be enhanced with the enhancement data (e.g., filters) described above. The rear-facing camera can, for example, be used to capture still images and videos in a more traditional camera mode, where the images are similarly enhanced using the enhancement data. In addition to the front-facing camera and the rear-facing camera, client device 402 can also include a 360° camera for capturing 360° photos and videos.

[0201] Additionally, the camera system of the client device 402 may include dual rear cameras (e.g., a main camera and a depth sensing camera), or even triple, quad, or quintuple rear camera configurations on the front and back sides of the client device 402. For example, these multi-camera systems may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, and a depth sensor.

[0202] The location component 936 includes a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure, from which the altitude can be obtained), an orientation sensor component (e.g., a magnetometer), and the like.

[0203] A variety of technologies may be used to achieve communication. The I / O component 902 also includes a communication component 938 that is operable to couple the machine 900 to the network 922 or device 924 via a corresponding coupling or connection. For example, the communication component 938 may include a network interface component or another suitable device that interfaces with the network 922. In other examples, the communication component 938 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, Components (e.g. Low energy consumption), Components and other communication components that provide communication via other modalities. Device 924 can be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0204] In addition, the communication component 938 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 938 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional bar codes such as Universal Product Code (UPC) bar codes, multi-dimensional bar codes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, UltraCode, UCC RSS-2D bar codes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 938, such as a location obtained via Internet Protocol (IP) geolocation, ... Location derived from signal triangulation, location derived via detection of NFC beacon signals that can indicate a specific location, etc.

[0205] Various memories (e.g., main memory 914, static memory 916, and memory of processor 904) and storage unit 918 may store one or more sets of instructions and data structures (e.g., software) used by or implementing any one or more of the methods or functions described herein. When executed by processor 904, these instructions (e.g., instructions 910) cause various operations to implement the disclosed examples.

[0206] The instructions 910 may be sent or received via a network interface device (e.g., a network interface component included in the communication component 938) using a transmission medium and using any of several well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)) over the network 922. Similarly, the instructions 910 may be sent or received via a coupling (e.g., a peer-to-peer coupling) to the device 924 using a transmission medium.

[0207] Software Architecture

[0208] Fig.101000 is a block diagram illustrating a software architecture 1004 that may be installed on any one or more of the devices described herein. The software architecture 1004 is supported by hardware such as a machine 1002 including a processor 1020, a memory 1026, and an I / O component 1038. In this example, the software architecture 1004 may be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 1004 includes layers such as an operating system 1012, a library 1010, a framework 1008, and an application 1006. In operation, the application 1006 invokes an API call 1050 through the software stack and receives a message 1052 in response to the API call 1050.

[0209] The operating system 1012 manages hardware resources and provides public services. The operating system 1012 includes, for example, a kernel 1014, services 1016, and drivers 1022. The kernel 1014 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 1014 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. Services 1016 can provide other public services to other software layers. Drivers 1022 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 1022 may include near-eye display drivers, camera drivers, or Low-power drivers, Flash drivers, Serial communication drivers (e.g., USB drivers), Drivers, audio drivers, power management drivers, etc.

[0210] The library 1010 provides a common low-level infrastructure used by the application 1006. The library 1010 may include a system library 1018 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 1010 may include an API library 1024, such as a media library (e.g., a library for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for presentation in two dimensions (2D) and three dimensions (3D) in graphics content on a near-eye display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. The library 1010 may also include various other libraries 1028 to provide many other APIs to the application 1006 .

[0211] The framework 1008 provides a common high-level infrastructure used by the applications 1006. For example, the framework 1008 provides various graphical user interface (GUI) functions, high-level resource management, and high-level positioning services. The framework 1008 can provide a wide range of other APIs that can be used by the applications 1006, some of which can be specific to a particular operating system or platform.

[0212] In an example, applications 1006 may include a home application 1036, a contacts application 1030, a browser application 1032, a book reader application 1034, a positioning application 1042, a media application 1044, a messaging application 1046, a game application 1048, and a variety of other applications such as third-party applications 1040. Applications 1006 are programs that execute functions defined in the program. Various programming languages ​​may be used to create one or more of the applications 1006 constructed in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C or assembly language). In a specific example, third-party applications 1040 (e.g., provided by an entity other than the vendor of a particular platform using ANDROID TM or IOS TM Software Development Kit (SDK) can be used for mobile operating systems, such as IOS TM ANDROID TM , Phone, or other mobile operating system running on the mobile software. In this example, the third-party application 1040 can activate the API call 1050 provided by the operating system 1012 to facilitate the functions described in this article.

[0213] Glossary

[0214] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media that facilitates communication of such instructions. Instructions may be sent or received over a network using a transmission medium via a network interface device.

[0215] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.

[0216] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, other type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0217] "Component" refers to a device, a physical entity, or logic with boundaries defined by function or subroutine calls, branch points, APIs, or other technologies that provide partitioning or modularization for specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be packaged functional hardware units designed for use with other components and can be part of a program that generally performs specific functions in related functions. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., independent computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations described herein. Hardware components can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include a dedicated circuit or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component may also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It should be understood that it can be decided whether to mechanically implement the hardware component in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., configured by software) circuit system for cost and time considerations. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an example in which a hardware component is temporarily configured (e.g., programmed), it is not necessary to configure or instantiate each of the hardware components at any one time. For example, in the case where a hardware component includes a general-purpose processor that is configured by software to become a special-purpose processor, the general-purpose processor can be respectively configured as different special-purpose processors (e.g., including different hardware components) at different times. Software accordingly configures one or more specific processors to, for example, constitute a specific hardware component at one time, and to constitute different hardware components at different times. Hardware components can provide information to other hardware components and receive information from other hardware components. Accordingly, the described hardware components can be considered to be coupled in communication.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between or among two or more hardware components. In examples where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessible to multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to it in communication. Then, another hardware component can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., a collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether it is temporarily configured or permanently configured, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described in this article can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors or the components implemented by the processor. In addition, one or more processors can also operate to support the execution of related operations in a "cloud computing" environment or as a "software as a service" (SaaS) operation. For example, at least some of the operations in the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., API). The execution of some operations in the operation can be distributed between processors, can not only reside in a single machine, but also can be deployed across several machines. In some examples, a processor or a component implemented by a processor can be located in a single geographical location (e.g., in a home environment, an office environment, or a server cluster). In other examples, a processor or a component implemented by a processor can be distributed across several geographical locations.

[0218] "Computer-readable storage media" refers to both machine storage media and transmission media. Therefore, these terms include storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.

[0219] "Transient messages" are messages that are accessible for a limited duration. Transient messages can be text, images, videos, etc. The access time for transient messages can be set by the sender of the message. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is transient.

[0220] "Machine storage media" refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage media", "device storage media", and "computer storage media" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage media", "computer storage media", and "device storage media" expressly exclude carrier waves, modulated data signals, and other such media (at least some of which are encompassed by the term "signal media").

[0221] “Non-transitory computer-readable storage medium” refers to tangible media capable of storing, encoding, or carrying instructions to be executed by a machine.

[0222] "Signal medium" refers to any intangible medium capable of storing, encoding or carrying instructions executed by a machine and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signals, carrier waves, etc. The term "modulated data signal" refers to a signal in which one or more of its various characteristics is set or changed in a manner that encodes information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and can be used interchangeably in the present disclosure. A device may include a lens assembly. A device may include an eyeglass body on which a lens assembly is mounted, the eyeglass body being configured for head-mounted wear, during which the lens assembly is supported in an appropriate position to occupy the wearer's field of view so that the wearer's perspective of the surrounding environment is through the lens assembly. A device may include a display mechanism configured to provide a near-eye display to the wearer that coincides with the lens assembly. A device may include a shielding mechanism contained in a lens assembly, and the shielding mechanism is configured to selectively set the lens assembly between at least two different optical states, wherein the at least two different optical states include: a see-through state, in which the lens assembly is functionally transparent so that a near-eye display is superimposed on a view of the surrounding environment provided by external light passing through the lens assembly; and an opaque state, in which the lens assembly is functionally opaque to ambient light, blocking ambient backlighting of the near-eye display.

Claims

1. A device comprising: Lens assembly; An eyeglass body on which the lens assembly is mounted, the eyeglass body being configured for head-mounted wear, wherein during head-mounted wear, the lens assembly is supported in a suitable position to occupy the wearer's field of view so that the wearer's perspective of the surrounding environment passes through the lens assembly; a display mechanism configured to provide a near-eye display to the wearer that is coincident with the lens assembly; as well as a shielding mechanism included in the lens assembly and configured to selectively set the lens assembly between at least two different optical states, the at least two different optical states comprising: a see-through state in which the lens assembly is functionally transparent such that the near-eye display is superimposed on a view of the surrounding environment provided by external light passing through the lens assembly; as well as An opaque state in which the lens assembly is functionally opaque to ambient light, blocking ambient backlighting of the near-eye display.

2. The device according to claim 1, wherein: The shielding mechanism is configured to place the lens assembly in a blocked state, in which the lens assembly is completely opaque and substantially blocks any ambient light from passing through the lens assembly.

3. The device according to claim 1, wherein: The shielding mechanism is configured to enable graded variation of the opacity of the lens assembly, thereby enabling controlled graded variation of the intensity of the ambient backlight of the near-eye display.

4. The device according to claim 3, wherein: The shielding mechanism allows for selective user-controlled, graduated variation of the opacity of the lens assembly.

5. The device according to claim 3, wherein: The shielding mechanism is configured such that an objective intensity of the near-eye display is not affected by changes in opacity of the lens assembly.

6. The device according to claim 1, wherein: The shielding mechanism includes a pair of stacked polarizers contained in the lens assembly, and the pair of polarizers can be selectively moved relative to one polarizer to modify the composite opacity of the pair of polarizers by changing the relative orientation of the corresponding polarization axes of the stacked polarizers, thereby achieving the switching of the lens assembly between the perspective state and the opaque state.

7. The device according to claim 6, wherein: The pair of stacked polarizers comprises: a static polarizer fixed in position relative to the eyeglass body; and A movable polarizer is configured to be selectively movable relative to the eyeglass body about an axis of rotation that is substantially aligned with an operative viewing direction through the lens assembly.

8. The device according to claim 7, wherein: The static polarizer is integrated with a primary optical element, which forms part of the lens assembly and carries the near-eye display, and the movable polarizer is located in front of the primary optical element, away from the wearer of the device, so that in the wearer's field of view, the near-eye display is located in front of the movable polarizer.

9. The device according to claim 8, wherein: The movable polarizer is manually adjustable in a rotational orientation relative to the rotation axis.

10. The device according to claim 8, wherein: The shielding mechanism also includes an actuator configured to effectuate actuated modification of a rotational orientation of the movable polarizer relative to the static polarizer.

11. The device according to claim 10 further includes a selective control mechanism, which is configured to receive a mode switching command provided by a user and, in response to the mode switching command, switch the lens assembly between the opaque state and the see-through state by driving movement of the movable polarizer through operation of the actuator.

12. The device of claim 1, further comprising an opacity controller, the opacity controller comprising one or more computer processor devices housed by the eyeglass body, the one or more computer processor devices configured to perform dynamic autonomous mode switching by performing the following operations, the operations comprising: continuously receiving current sensor data captured by one or more sensors included in the eyeglass body; as well as Based at least in part on the current sensor data, the lens assembly is autonomously switched from one of the see-through state and the opaque state to the other state.

13. The device according to claim 12, wherein: The opacity controller is configured to provide automatic control of the shielding mechanism by performing the following operations, the operations comprising: When the lens assembly is in a first one of its optical states, identifying satisfaction of a predefined mode switching criterion applicable to the first optical state; and In response to satisfying an applicable mode switching criterion, setting of the lens assembly from the first optical mode in the optical state of the lens assembly to a different second optical mode is automatically effected.

14. The device according to claim 13, wherein: The mode switching criteria for automatically switching the lens assembly from the see-through state to the opaque state include a combination of: The wearer of the device is in a non-walking state; as well as A selection input is received to render non-AR visual content via the near-eye display.

15. The apparatus according to claim 13, wherein: Mode switching criteria suitable for switching the lens assembly from the see-through state to the opaque state include recognizing that the wearer has entered a sleeping state.

16. The apparatus according to claim 12, wherein: The one or more computer processor devices are configured to trigger an autonomous mode switch based at least in part on the current sensor data by using a trained machine learning model.

17. A method comprising: accessing operational data from an eyewear device having an optical system with supporting electronics, the optical system being switchable between different optical states, the different optical states comprising a see-through state and an opaque state in which a wearer's view of their environment is obstructed by the optical system, the operational data comprising a current optical state of the optical system and sensor data captured by one or more sensors integrated in the eyewear device; identifying, when the optical state is in a first one of its optical states, satisfaction of a predefined mode switching criterion applicable to the first optical state; as well as In response to satisfying the mode switching criterion, the optical system is automatically switched from the first optical state mode to a different second optical state mode in the optical states of the optical system.

18. The method according to claim 17, wherein: The predefined mode switching criteria applicable when the optical assembly is in the see-through state includes identifying that the wearer has fallen asleep.

19. The method according to claim 17, wherein: The predefined mode switching criteria applicable when the optical assembly is in the see-through state includes identifying that the wearer is in a non-walking state.

20. A non-transitory computer-readable storage medium comprising instructions that, when executed by one or more computer processor devices, configure the one or more computer processor devices to perform operations comprising: accessing operational data from an eyewear device having an optical system with supporting electronics, the optical system being switchable between different optical states, the different optical states comprising a see-through state and an opaque state in which a wearer's view of their environment is obstructed by the optical system, the operational data comprising a current optical state of the optical system and sensor data captured by one or more sensors integrated in the eyewear device; identifying, when the optical system is in a first one of its optical states, satisfaction of a predefined mode switching criterion applicable to the first optical state; as well as In response to the mode switching criterion being met, the optical system is automatically set from the first optical state mode to a different second optical state mode among the optical states of the optical system.

Images