Lock screen for head-mounted augmented reality device

Abstract

A method of activating a head-mounted device is disclosed. The method includes activating the head-mounted device and performing a partial boot-up. Upon receiving an instruction to execute a function, it is determined whether the instruction is allowed for partial boot execution. The instruction is executed based on the instruction being allowed for partial boot execution, and boot of the head working device is completed based on the instruction not being allowed for partial boot execution. The method may also include determining whether the instruction requires user authentication to be executed, and executing the instruction based on the instruction being compatible with the partial launch and not requiring user authentication.

Description

[0001] priority

[0002] This application claims the benefit of priority to U.S. patent application Ser. No. 17 / 660,359, filed on Apr. 22, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to head-mounted devices with displays for augmented reality or virtual reality. Background Art

[0004] A head mounted device may be implemented with a transparent or translucent display through which a user of the head mounted device can view the surrounding environment. Such a device enables the user to look through the transparent or translucent display to view the surrounding environment, and also to see objects (e.g., virtual objects such as 3D renderings, images, videos, text, etc.) generated for display that appear to be part of the surrounding environment and / or superimposed on the surrounding environment. This is generally referred to as "augmented reality."

[0005] The head mounted device may also completely obstruct the user's field of view and display a virtual environment through which the user can move or be moved. This is often referred to as "virtual reality". As used herein, unless the context indicates otherwise, the term "augmented reality" or "AR" refers to both augmented reality and virtual reality as conventionally understood.

[0006] A user of the head mounted device may access a messaging application or a social networking application to view content or share content with other users of the application. In some cases, real-time content or stored content may be viewed and enhanced or modified by the user. That is, images, videos, or other media for enhancement may be captured from a real-time camera or may be retrieved from a local or remote data storage device.

[0007] In many cases, a head mounted device may be associated with a mobile device running an associated application. The mobile device often includes a lock screen for security that is activated when the device enters an inactive or less active state, such as based on the expiration of a timer or based on a user pressing a button to turn off the display of the mobile device. Current lock screens implemented on mobile devices can provide challenges when pairing with a device such as a head mounted device. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] To easily identify the discussion of any particular element or act, the highest digit or digits in a reference number refer to the figure number in which the element is first introduced.

[0009] Figure 1 is a perspective view of a head-mounted device including glasses according to some examples.

[0010] Figure 2 According to some examples Figure 1 Additional views of the head-mounted device.

[0011] Figure 3 It is shown that according to some examples, Figure 1 A block diagram of the networking system with details of the head mounted device.

[0012] Figure 4 is a block diagram illustrating the architecture of the glasses in more detail according to some examples.

[0013] Figure 5A and Figure 5B A sequence diagram for lock screen initialization when glasses start up according to some examples is shown.

[0014] Fig. 6A and Figure 6B A sequence diagram for user unlocking of glasses according to some examples is shown.

[0015] Fig. 7A and Figure 7B A sequence diagram for setting a PIN on unsafe glasses is shown according to some examples.

[0016] Figure 8 A sequence diagram for changing settings on safety glasses is shown according to some examples.

[0017] Fig. 9 A sequence diagram for clearing a PIN on safety glasses is shown according to some examples.

[0018] Fig. 10A and Fig. 10B A sequence diagram for pairing a client device with glasses is shown according to some examples.

[0019] Fig.11A and Fig. 11B A sequence diagram related to security and boot states of glasses and associated authorizations according to some examples is shown.

[0020] Fig.12 A sequence diagram for PIN entry on safety glasses is shown according to some examples.

[0021] Fig.13 A lock screen UI is shown for use on glasses in some examples.

[0022] Fig.14 is a flow diagram illustrating operations performed by glasses and a client device according to some examples.

[0023] Fig.15is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, according to some examples.

[0024] Fig.16 is a block diagram illustrating a software architecture within which the present disclosure may be implemented, according to some examples.

[0025] Fig.17 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed, for causing the machine to perform any one or more of the methodologies discussed. DETAILED DESCRIPTION

[0026] Known head-mounted devices such as AR glasses include a transparent or translucent display, which enables a user to look through the transparent or translucent display to view the surrounding environment. Additional information or objects (e.g., virtual objects, such as 3D renderings, images, videos, text, etc.) are displayed on the display and appear as part of the surrounding environment and / or overlaid on the surrounding environment to provide an augmented reality experience for the user. The display may, for example, include a waveguide that receives a beam from a projector, but any suitable display for presenting augmented or virtual content to the wearer may be used.

[0027] In some cases, navigation of information or a user interface of the head mounted device may be provided by voice commands or by input to an associated mobile device such as a smart phone. In some examples, a touch pad is provided on the head mounted device that can be used to provide xy touch and tap input to the head mounted device. Complex head mounted devices can now perform or initiate tasks previously associated with smart phones, such as browsing, viewing and capturing media, selecting recipients for captured media, and forwarding captured media to selected recipients. Providing appropriate and coordinated lock screen functionality with an associated smart phone can be challenging.

[0028] Head-mounted devices are also resource-constrained devices (e.g., in terms of memory, storage, and battery power), and are also latency-sensitive to ensure a satisfactory user experience. Head-mounted devices need to operate in a variety of environments in an efficient manner in terms of resource consumption and battery use. Memory usage may also fluctuate, such as when a user is capturing or forwarding media. A critical operation is system booting for the head-mounted device and associated mobile device, which can be associated with increased resource consumption and system stress caused by the operating system starting multiple services simultaneously. This can cause a variety of undesirable side effects, such as the system running out of memory or ending up in an unresponsive state.

[0029] Due to the limitations of user interaction with the head-mounted device, a large portion of the user interface is executed on the paired mobile device. In addition, there are multiple processes and processors operating in both the head-mounted device and the paired mobile device. The security state needs to be synchronized between the paired devices and processors. The main processor of the head-mounted device is also often turned off, which brings further challenges to boot time, security and encryption.

[0030] To address these technical issues, in some examples, security settings are modifiable on the mobile device, while the implementation of the security settings occurs on the head-mounted device. The secure software architecture enables security settings to be changed on the mobile device and kept synchronized between multiple processors. By dividing the data into user-sensitive data and non-sensitive data, the restart of the main processor of the head-mounted device after a period of inactivity is simplified. By enabling related services to operate in this limited functional mode, all functions that do not require user-sensitive data are available with reduced latency. Then, after the user unlocks the combination of the head-mounted device and the mobile device, the head-mounted device gracefully switches to full operational mode.

[0031] In some examples, a method performed by one or more processors in a head mounted device including one or more display devices is provided, the method comprising: receiving an activation input, performing a partial startup of the head mounted device, receiving an instruction for performing a function, determining whether the instruction is allowed for partial startup execution, executing the instruction based on the instruction being allowed for partial startup execution, and completing the startup of the head mounted device based on the instruction not being allowed for partial startup execution. The method may also include: determining whether the instruction requires user authentication to be executed, and executing the instruction based on the instruction being partially startup compatible and not requiring user authentication. The partial startup of the head mounted device may include turning on a lock screen service and a camera service.

[0032] In addition, the method may include determining whether the instruction requires user authentication to be executed, and based on the instruction requiring user authentication, performing a user authentication process regardless of whether the instruction is partially launch compatible. The method may also include verifying correct user authentication, completing launch of the head mounted device, and executing the instruction.

[0033] Verifying correct user authentication may include receiving a password from the associated mobile device. Based on the correct user authorization not being verified using the password received from the associated mobile device, the method may also include prompting the user to enter the password via the head mounted device. In addition, the authentication state of the head mounted device may be maintained based on the proximity of the head mounted device to the associated mobile device.

[0034] In some examples, a head mounted device is provided, which includes one or more cameras, one or more display devices, and one or more processors. The head mounted device also includes a memory storing instructions, which when executed by the one or more processors configures the device to perform operations according to any of the methods and limitations described above, including but not limited to: receiving an activation input, performing a partial startup of the head mounted device, receiving an instruction for performing a function, determining whether the instruction is allowed for partial startup execution, executing the instruction based on the instruction being allowed for partial startup execution, and completing the startup of the head mounted device based on the instruction not being allowed for partial startup execution.

[0035] In some examples, a non-transitory computer-readable storage medium is provided that includes instructions that, when executed by a head-mounted device that includes one or more display devices and one or more cameras, cause the head-mounted device to perform operations according to any of the methods and limitations described above, including but not limited to: performing partial startup of the head-mounted device, receiving instructions for performing a function, determining whether the instruction is allowed for partial startup execution, executing the instruction based on the instruction being allowed for partial startup execution, and completing startup of the head-mounted device based on the instruction not being allowed for partial startup execution.

[0036] Other technical features may be very obvious to those skilled in the art from the following drawings, descriptions and claims.

[0037] Figure 1 1 is a perspective view of a head mounted device (e.g., glasses 100) according to some examples. The glasses 100 may include a frame 102, which is made of any suitable material, such as plastic or metal, including any suitable shape memory alloy. In one or more examples, the frame 102 includes a first optical element holder or a left optical element holder 104 (e.g., a display or lens holder) and a second optical element holder or a right optical element holder 106 connected by a bridge 112. A first optical element or a left optical element 108 and a second optical element or a right optical element 110 may be disposed in the left optical element holder 104 and the right optical element holder 106, respectively. Each of the right optical element 110 and the left optical element 108 may be a lens, a display, a display component, or a combination of the foregoing. Any suitable display component may be disposed in the glasses 100.

[0038] The frame 102 additionally includes a left arm or temple piece 120 and a right arm or temple piece 122. In some examples, the entire frame 102 can be formed from a single piece of material to have a unitary or unitary construction.

[0039] The glasses 100 may include a computing device such as a computer 118, which may be of any suitable type to be carried by the frame 102, and in one or more examples, the computing device may be of a suitable size and shape to be at least partially disposed in one of the temple pieces 120 and temple pieces 122. The computer 118 may include one or more processors as well as memory, wireless communication circuitry, and a power source. As discussed below, the computer 118 includes a low-power circuitry, a high-speed circuitry, and a display processor. Various examples may include these elements in different configurations or integrated together in different ways. Additional details of various aspects of the computer 118 may be implemented as shown in the data processor 302 discussed below.

[0040] The computer 118 additionally includes a battery 116 or other suitable portable power supply. In some examples, the battery 116 is disposed in the left temple piece 120 and is electrically coupled to the computer 118 disposed in the right temple piece 122. The glasses 100 may include a connector or port (not shown) suitable for charging the battery 116, a wireless receiver, transmitter or transceiver (not shown), or a combination of such devices.

[0041] The glasses 100 include a camera 114. Although two cameras are depicted, some examples contemplate the use of a single or additional (i.e., more than two) cameras. In one or more examples, the glasses 100 include any number of input sensors or other input / output devices in addition to the camera 114. Such sensors or input / output devices may additionally include biometric sensors, position sensors, motion sensors, etc.

[0042] The glasses 100 may also include a touchpad 124 mounted to or integrated with one or both of the left temple piece 120 and the right temple piece 122. The touchpad 124 is generally arranged vertically, with the touchpad 124 being approximately parallel to the temple of the user in some examples. As used herein, generally vertical alignment means that the touchpad is at least more vertical than horizontal, although preferably more vertical than the vertical. Additional user input may be provided by one or more buttons 126, which in the illustrated embodiment are disposed on the outer upper edges of the left optical element holder 104 and the right optical element holder 106. The one or more touchpads 124 and buttons 126 provide a device by which the glasses 100 can receive input from the user of the glasses 100.

[0043] Figure 2 The eyeglasses 100 are shown from the wearer's perspective. Figure 1Several elements shown in FIG. have been omitted. Figure 1 As described, Figure 2 The illustrated eyeglasses 100 include a left optical element 108 and a right optical element 110 secured within each of a left optical element holder 104 and a right optical element holder 106, respectively.

[0044] The glasses 100 include a front optical component 202 including a right projector 204 and a right near-eye display 206 , and a front optical component 210 including a left projector 212 and a left near-eye display 216 .

[0045] In one embodiment, the near eye display is a waveguide. The waveguide includes a reflective structure or a diffractive structure (e.g., a grating and / or an optical element such as a mirror, lens, or prism). Light 208 emitted by projector 204 encounters the diffractive structure of the waveguide of near eye display 206, which directs the light toward the right eye of the user to provide an image on or in the right optical element 110 that is superimposed on the view of the real world seen by the user. Similarly, light 214 emitted by projector 212 encounters the diffractive structure of the waveguide of near eye display 216, which directs the light toward the left eye of the user to provide an image on or in the left optical element 108 that is superimposed on the view of the real world seen by the user.

[0046] However, it should be understood that other display technologies or configurations can be provided that can display images to a user in a forward field of view. For example, instead of providing a projector 204 and a waveguide, an LCD, LED or other display panel or surface can be provided instead.

[0047] In use, information, content, and various user interfaces will be presented to the wearer of the glasses 100 on the near-eye display. As described in more detail below, in addition to being displayed on associated devices such as Figure 3 In addition to providing voice input or touch input on the client device 328 as shown, the user can then interact with the glasses 100 using the touchpad 124 and / or buttons 126.

[0048] Figure 3 is a block diagram illustrating a networked system 300 including details of glasses 100 according to some examples.

[0049] The networked system 300 includes the glasses 100, a client device 328, and a server system 332. The client device 328 may be a smartphone, a tablet computer, a tablet phone, a laptop, an access point, or any other such device capable of connecting to the glasses 100 using both a low power wireless connection 336 and a high speed wireless connection 334. The client device 328 is connected to the server system 332 via a network 330. The network 330 may include any combination of wired and wireless connections. The server system 332 may be one or more computing devices that are part of a service or network computing system. The client device 328 and any elements of the server system 332 and the network 330 may use Fig.16 and Fig.17 The details of the software architecture 1604 or machine 1700 described in are implemented.

[0050] The glasses 100 include a data processor 302, a display 310, one or more cameras 308, and additional input / output elements 316. The input / output elements 316 may include a microphone, an audio speaker, a biometric sensor, an additional sensor, or an additional display element integrated with the data processor 302. Fig.16 and Fig.17 Examples of input / output elements 316 are further discussed. For example, input / output elements 316 may include any I / O component 1706, including output component 1728, motion component 1736, etc. Figure 2 Examples of display 310 are discussed in . In the particular examples described herein, display 310 includes a display for each of the user's left eye and right eye.

[0051] Data processor 302 includes image processor 306 (eg, video processor), GPU and display driver 338, tracking module 340, interface 312, low power circuitry 304, and high speed circuitry 320. Components of data processor 302 are interconnected by bus 342.

[0052] The interface 312 refers to any source of user commands provided to the data processor 302. In one or more examples, the interface 312 is a physical button that, when pressed, sends a user input signal from the interface 312 to the low-power processor 314. The low-power processor 314 can process pressing such a button and then immediately releasing it as a request to capture a single image, and vice versa. The low-power processor 314 can process pressing such a button within a first time period as a request to capture video data when the button is pressed and stop video capture when the button is released, wherein the video captured when the button is pressed is stored as a single video file. Alternatively, pressing the button for a long period of time can capture a still image. In some examples, the interface 312 can be any mechanical switch or physical interface capable of accepting user input associated with a data request from the camera 308. In some examples, the interface 312 can have a software component, or can be associated with a command received wirelessly from another source, such as from the client device 328.

[0053] Image processor 306 includes circuitry for receiving signals from camera 308 and processing those signals from camera 308 into a format suitable for storage in memory 324 or for transmission to client device 328. In one or more examples, image processor 306 (e.g., a video processor) includes a microprocessor integrated circuit (IC) customized for processing sensor data from camera 308, and volatile memory used by the microprocessor in operation.

[0054] The low power circuit system 304 includes a low power processor 314 and a low power wireless circuit system 318. These elements of the low power circuit system 304 can be implemented as separate elements or can be implemented on a single IC as part of a single system on a chip. The low power processor 314 includes logic for managing other elements of the glasses 100. As described above, for example, the low power processor 314 can accept user input signals from the interface 312. The low power processor 314 can also be configured to receive input signals or command communications from a client device 328 via a low power wireless connection 336. The low power wireless circuit system 318 includes circuit elements for implementing a low power wireless communication system. Bluetooth TM Smart, also known as Bluetooth TM Low power consumption is a standard implementation of a low power wireless communication system that may be used to implement low power wireless circuitry 318. In some examples, other low power communication systems may be used.

[0055] High-speed circuit system 320 includes a high-speed processor 322, a memory 324, and a high-speed wireless circuit system 326. High-speed processor 322 can be any processor capable of managing high-speed communications and operations of any general-purpose computing system required by data processor 302. High-speed processor 322 includes the processing resources required to manage high-speed data transmission over high-speed wireless connection 334 using high-speed wireless circuit system 326. In some examples, high-speed processor 322 executes an operating system such as the LINUX operating system or a program such as Fig.16 The high-speed processor 322, which executes the software architecture of the data processor 302, manages data transmission with the high-speed wireless circuit system 326, in addition to any other duties. In a specific example, the high-speed wireless circuit system 326 is configured to implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 communication standard, which is also referred to herein as Wi-Fi. In some examples, the high-speed wireless circuit system 326 can implement other high-speed communication standards.

[0056] Memory 324 includes any storage device capable of storing camera data generated by camera 308 and image processor 306. Although memory 324 is shown as being integrated with high-speed circuitry 320, in some examples, memory 324 may be a separate, stand-alone element of data processor 302. In particular such examples, electrical wiring may provide a connection from image processor 306 or low-power processor 314 to memory 324 through a chip including high-speed processor 322. In some examples, high-speed processor 322 may manage addressing of memory 324 so that low-power processor 314 will direct high-speed processor 322 whenever a read or write operation involving memory 324 is needed.

[0057] The tracking module 340 estimates the pose of the glasses 100. For example, the tracking module 340 uses image data and corresponding inertial data from the camera 308 and the positioning component 1740 and GPS data to track the position and determine the pose of the glasses 100 relative to a reference frame (e.g., a real-world environment). The tracking module 340 continuously collects and uses updated sensor data describing the movement of the glasses 100 to determine an updated three-dimensional pose of the glasses 100, which indicates changes in relative position and orientation relative to physical objects in the real-world environment. The tracking module 340 allows the glasses 100 to visually place virtual objects relative to physical objects within the user's field of view via the display 310.

[0058] The GPU and display driver 338 can use the pose of the glasses 100 to generate frames of virtual content or other content to be presented on the display 310 when the glasses 100 are operating in a conventional augmented reality mode. In this mode, the GPU and display driver 338 generate updated frames of virtual content based on the updated three-dimensional pose of the glasses 100, which reflects changes in the user's position and orientation relative to physical objects in the user's real environment.

[0059] As further described herein, the virtual content presented on the display 310 can provide a virtual reality experience that includes or involves various image processing operations corresponding to image modification, filtering, media overlays, transformations, and the like. In some examples, these image processing operations provide an interactive experience of a real-world environment, in which objects, surfaces, backgrounds, lighting, and the like in the real world are enhanced by computer-generated perceptual information. In this context, an "augmented reality effect" includes a collection of data, parameters, and other assets for applying a selected augmented reality experience to an image or video feed. In some examples, the augmented reality effect is provided by Snap Inc. under the registered trademark LENSES.

[0060] One or more functions or operations described herein may also be performed in an application resident on the glasses 100 or on the client device 328 or on a remote server. For example, one or more functions or operations described herein may be performed by one of the applications 1606, such as the messaging application 1646.

[0061] Figure 4 is a block diagram showing the architecture of the glasses 100 in more detail according to some examples. Figure 3 As discussed, the glasses include high-speed circuitry 320 and low-power circuitry 304. High-speed circuitry 320 and low-power circuitry 304 may each be implemented as a system on a chip (SOC), for example.

[0062] High-speed circuit system 320 runs operating system 402 and secure OS 404. Secure OS 404 provides a trusted execution environment, which is a secure area of ​​high-speed circuit system 320 that ensures that sensitive data is stored, processed, and protected in an isolated and trusted environment. As such, secure OS 404 performs user authentication functions and provides protection against software attacks generated in or originating from operating system 402.

[0063] The secure OS 404 runs keymaster 406 and gatekeeper 410 security related functions. The keymaster 406 stores security certificates and hashed passwords, and generates keys and certificates for encrypting the file system used in the memory 324 of the high-speed circuit system 320. The gatekeeper 410 handles security requests from the operating system 402 and delegates related operations to the keymaster 406 during password setting, changing, and verification operations. In some examples, the password can be a personal identification number (PIN), although other user authentication methods such as biometrics can be used.

[0064] The gatekeeper 410 also implements a user lockout in the event that there are too many failed login attempts. A monotonic clock is used to measure the number of attempts and lockout time to avoid subverting the lockout implementation by changing the time or draining the battery of the glasses 100. When a password is first set on the glasses after a factory reset, the gatekeeper 410 encrypts the user file system with a key and associates it with the user ID. At the start of the glasses 100, the user must unlock the glasses 100 at least once to enable the operating system 402 to decrypt and mount the file system. Subsequent lockouts of the glasses 100 will not encrypt the file system again until a reboot.

[0065] Until the user has entered their password and the file system is decrypted and mounted after boot, the glasses 100 are said to be in direct boot mode. In direct boot mode, only certain direct boot aware applications can and are allowed to execute, and access to encrypted data storage is not provided. Therefore, direct boot mode is a partial bootup or boot mode in terms of applications allowed to be executed and memory accessible by direct boot aware applications. Direct boot mode is indicated by the LOCKED_BOOT_COMPLETED system status and system status message, and triggers the execution of direct boot aware applications. Once the user unlocks the glasses 100 using their password, the architecture completes the rest of the initialization, then updates the system status, and sends a BOOT_COMPLETED status message to launch several other applications, including the home / launcher application that is part of the glasses applications 412.

[0066] The operating system 402 includes a framework with several auxiliary classes and wrappers to communicate with the secure OS 404 and cache associated values. In some examples, the operating system 402 and the associated framework are based on the Android Open Source Project (AOSP). The operating system 402 includes a lock screen service 418, a lock settings service 420, an alarm management service 422, a property service 424, and a camera device service 428.

[0067] The lock screen service 418 is a direct boot-aware application and is started first to initialize the lock state 432 of the glasses 100 before any other service or application reads the lock state 432. The lock screen service 418 is also a persistent application because the operating system 402 automatically restarts the application whenever the lock screen service 418 terminates. The lock screen service 418 also maintains all user settings for security, such as the amount of time after the glasses 100 are removed (or "taken off") from the user's face before the glasses 100 are locked. For additional security, all setting values ​​are stored by the lock screen service 418 as encrypted strings in key-value pairs.

[0068] The lock screen service 418 also handles all password-related communications with the lock setting service 420 via the operating system 402 API, including processing the setting of passwords, changing passwords and removing passwords from the device, verifying the entered passwords and reporting successful and failed verification attempts, and reporting any user lock status and time to other services. The lock screen service 418 also uses the alarm management service 422 in the Android framework to handle setting the correct lock alarm. When the alarm sounds, the lock screen service 418 is notified by the alarm management service 422 and locks the glasses by setting the lock state attribute field 426 attribute in the attribute service 424 to true (True). The lock screen service 418 also clears any alarms when the setting or state changes (for example, if the user puts on the glasses again (or "puts on" the glasses 100) before the take-off alarm timer expires), and sets new alarms as needed.

[0069] The lock screen service 418 also monitors the state of the wear sensor 436 and notifies the relevant service when the user puts on or takes off the glasses 100. The lock screen service 418 queries the low power circuit system 304 for the state of the wear sensor 436 via the proprietary glasses message service 430.

[0070] The alarm management service 422 is a standard operating system 402 service that enables an application or service to set an alarm that provides it with a message at a specified later time. The alarm management service 422 is often used to initiate periodic or delayed tasks that otherwise have imprecise timers. This allows precise coordination with other tasks or services to improve efficiency. The lock screen service 418 requires precise alarms based on normal operating hours to avoid problems that might otherwise be caused by changed date and time settings (e.g., a specific time of day, date, or time zone).

[0071] All cryptographic and security-related communications between secure OS 404 and any other service or application on high-speed circuit system 320 are routed through lock setup service 420. Lock setup service 420 provides a wrapper for communicating with gatekeeper 410 and communicates with gatekeeper 410 using an inter-process communication protocol specific to secure OS 404.

[0072] The attribute service 424 is also a direct boot aware application. The attribute service 424 stores the current lock state 432 in the lock state attribute field 426 as a true or false (binary) value.

[0073] The glasses application 412 is a proprietary application that provides a user experience for the glasses 100. The glasses application 412 is the main foreground activity executed by the glasses 100, and it initializes and maintains an instance of an AR module (not shown), which manages and displays any selected AR content or effects and user interfaces provided by the glasses 100. Figure 4 As shown, the glasses application 412 also includes a lock screen UI 416 and a lock screen controller 414. Since the lock screen UI 416 and the lock screen controller 414 display the lock screen to the user, these are also direct boot aware services that are started in direct boot mode and initialized before the rest of the glasses application 412.

[0074] The lock screen UI 416 displays a lock screen to the user on the display 310 of the glasses 100 and receives user input of a PIN for unlocking the glasses 100. The lock screen UI 416 also displays other security-related information such as a lock timer, animations, or dialog boxes to indicate failed attempts and the number of failed attempts, etc.

[0075] The lock screen controller 414 monitors the lock state 432 maintained by the attribute service 424. When the glasses are locked and the lock state 432 has a true value, the lock screen controller 414 prevents the user and the system from accessing any other glasses 100 functions. The lock state 432 of the glasses 100 is communicated to the user via the lock screen UI 416 as needed. The lock screen controller 414 also communicates with the lock screen service 418 via the glasses messaging service 430 to pass the PIN entered by the user to the lock screen service 418. The lock screen controller 414 also initiates the main UI of the glasses 100 based on the lock state 432 changing from true to false when the unlock attempt is successful or the device is unlocked via other means such as a system reset or alarm expiration.

[0076] The proprietary glasses messaging service 430 is a request-response protocol that handles remote procedure calls (RPCs) between the messaging client 408 on the client device 328, the Bluetooth controller 434 in the low-power circuit system 304, and the lock screen controller 414 in the glasses application 412. All software modules proprietary to the manufacturer of the glasses 100 and the provider of the messaging client 408 communicate with each other using the glasses messaging service 430. The glasses messaging service 430 also acts as a controller to transmit messages between the wear sensors 436 and the lock screen service 418.

[0077] In some examples, the glasses messaging service 430 supports the following remote procedure calls.

[0078] ●SetUserDeviceSecurityRequest: This RPC is sent by the messaging client 408 to set a password on unsecured glasses 100 that do not have an existing password, remove the password and all security settings (including user settings) on glasses 100 that have an existing password, thereby disabling security, or changing settings such as the delay before unlocking, the events that trigger locking, etc.

[0079] GetUserDeviceSecurityRequest: This RPC returns all user settings for the glasses 100 as well as the lock state 432 status (eg, device locked, timed out, etc.) This RPC is used by both the messaging client 408 and the lock screen controller 414 to query the current lock state 432 status during an initialization or refresh operation.

[0080] ●SetUserDevicePasswordRequest: This RPC is used by the messaging client 408 to change the password of the glasses 100.

[0081] IsDeviceSecure: This RPC is used at initial startup to determine if a password has been set for the glasses 100.

[0082] ●VerifyPasscodeRequest: This RPC is used to verify the password and, if the password is verified, authorize the specified request. The following requests are supported: a) User Unlock: Verify the password and, if successful, unlock the glasses 100; b) Proximity Unlock: Verify the password and, if successful, the device is unlocked or remains unlocked when the client device 328 and the messaging client 408 are connected to the glasses 100 via Bluetooth or other short-range links; c) Verify: Only used to verify the password, which the messaging client 408 can use to verify the current PIN during pairing with the glasses 100 or when changing the password.

[0083] ● PairingRequest: The low power circuitry 304 firmware uses this RPC to verify that the glasses are secure to pair via Bluetooth. This is only allowed when the operating system 402 is fully booted and the lock state 432 is false (ie, the glasses 100 are unlocked).

[0084] The low power circuitry 304 includes a Bluetooth controller 434 that handles Bluetooth communications with the messaging client 408 .

[0085] Figure 5A and Figure 5BA sequence diagram is shown for lock screen initialization at startup of glasses 100 according to some examples. The sequence diagram shows the information flow and calls between components and the associated state changes.

[0086] The sequence diagram begins when the operating system 402 completes the locked boot process and reports that the locked boot process is complete. This is broadcast as a system status message. In response to the locked boot completion status message, the lock screen service 418 is started at operation 502. The lock screen service 418 then queries the operating system 402 (via the lock setting service 420) as to whether the glasses 100 are secure. If a password has been set, the glasses 100 are considered a secure device. If the lock setting service 420 responds with true, the lock screen service 418 (via the lock setting service 420) sets the lock state to true to lock the glasses 100. If the lock setting service 420 responds with false, the lock screen service 418 sets the lock state to false to unlock the glasses 100.

[0087] In response to the lock boot complete status message, the camera service 428 is started at operation 504. The camera service 428 is a boot awareness process. The glasses application 412 is also started at operation 506. Then, in operation 508, the glasses application 412 initializes the lock screen controller 414.

[0088] Then, the glasses application 412 requests the lock state 432 from the lock screen service 418. Figure 5B 412. The operating system 402 continues with a true or false reply from the glasses application 412. If the operating system 402 reports to the glasses application 412 that the device is locked (true), the lock screen UI 416 is launched in operation 510. Assuming that the unlocking is successful in operation 512, the glasses application 412 launches the full user interface of the glasses 100 in operation 514. If the operating system 402 reports to the glasses application 412 that the device is unlocked (false), the full user interface of the glasses 100 is launched in operation 514.

[0089] Then, in operation 516, the operating system 402 completes the boot process and broadcasts a boot complete system message. At this point, the glasses 100 are fully booted and functional.

[0090] Fig. 6A and Figure 6B A sequence diagram is shown for a user to unlock the glasses 100 according to some examples. The sequence diagram shows the information flow and calls between components and the associated state changes.

[0091] Fig. 6AThe sequence diagram shown in FIG6 begins with the lock screen UI 416 receiving user input of a PIN in operation 602. The lock screen UI 416 then sends the PIN along with a verify PIN request to the lock screen controller 414. The lock screen controller 414 in turn sends the PIN along with the verify PIN request to the lock screen service 418. The lock screen service 418 in turn sends the PIN along with the verify PIN request to the lock settings service 420. The lock settings service 420 in turn sends the PIN along with the verify PIN request to the lock screen controller 414.

[0092] If the PIN is correct, the secure OS 404 replies to the lock settings service 420 with a success message, which is passed by the lock settings service 420 to the lock screen service 418. In response, the lock screen service 418 instructs the lock settings service 420 to set the lock state to false, which indicates that the glasses 100 are in an unlocked state. The lock screen service 418 then passes the success message to the lock screen controller 414, which dismisses the lock screen UI 416 and launches the main user interface of the glasses in operation 604.

[0093] Figure 6B The sequence diagram shown in FIG. 6 also includes receiving user input of a PIN by the lock screen UI 416 in operation 606 and sending the PIN and the verification PIN request along the path of the lock screen UI 416 to the user input of the PIN. Fig. 6A 404. However, in this case, the PIN is incorrect, and the secure OS 404 responds with a failure message to the lock setup service 420. The message is passed down the line from the lock setup service 420 to the lock screen service 418, to the lock screen controller 414, and then to the lock screen UI 416, where, in operation 608, the glasses 100 display a PIN input error animation to the user.

[0094] Fig. 7A and Figure 7B A sequence diagram for setting a PIN on unsecured glasses 100 according to some examples is shown. The sequence diagram shows the information flow and calls between components and the associated state changes. In some examples, this sequence can occur when a user of new glasses 100 receives after pairing.

[0095] Fig. 7A4. The sequence is shown in the case where a password has not been set. The sequence diagram begins with the messaging client 408 requesting a device security state from the lock screen service 418. This is accomplished via the glasses messaging service 430, the low power circuit system 304, and the operating system 402 on the client device 328. In this particular case, the lock screen service 418 replies to the messaging client 408 with an unsecured security state. The messaging client 408 then prompts the user to enter a new PIN, which is entered at operation 702. The messaging client 408 then sends a set PIN message to the lock screen service 418, which in turn passes it to the lock setup service 420, which in turn passes it to the secure OS 404. If this is the first time a PIN is set, the secure OS 404 turns on file-based encryption at operation 704 to further protect the device.

[0096] The security OS 404 then replies to the lock setup service 420 with a success message or a failure message, which is passed along the line to the messaging client 408. In the case of a failure message, an appropriate failure dialog box or animation is presented by the messaging client 408 on the client device 328.

[0097] In the case of a success message, the PIN is stored in the keychain of the messaging client 408 in operation 708. Additionally, if applicable, such as if this is the first time a PIN is set on the glasses 100, the new settings are stored in the lock screen service 418 in operation 706. For example, these settings may be the amount of time after the glasses 100 are taken off, or the loss of proximity between the glasses 100 and the client device 328 to which the glasses are locked. Additionally, the lock screen service 418 may set an alert with the alert management service 422 in operation 710 if the new settings require or are specified by the new settings.

[0098] Fig. 7A 418. The sequence is shown in the case where a password has not been set. The sequence diagram begins with the messaging client 408 requesting the device security state from the lock screen service 418. In this particular case, the lock screen service 418 replies to the messaging client 408 with a secure security state. The messaging client 408 then receives input of a new PIN from the user in operation 712 and retrieves the current PIN from the key chain in operation 714. The messaging client 408 then sends a set security state message to the lock screen service 418, which includes both the new PIN and the current PIN. The lock screen service 418 then sends a verification PIN message to the lock settings service 420 along with the old PIN, and the lock settings service 420 sends the verification PIN message to the security OS 404.

[0099] The security OS 404 checks the old PIN against the PIN stored in the key master 406 and sends a success message or a failure message back to the lock setup service 420. The failure message will be passed back along the line to the messaging client 408, and the messaging client 408 will present an appropriate failure dialog box or animation on the client device 328.

[0100] A success message from the secure OS 404 to the lock setup service 420 in response to the request to verify the old PIN is passed by the lock setup service 420 to the lock screen service 418. The lock screen service 418 then sends a set new PIN message to the lock setup service 420, which in turn sends a set new Pin message to the secure OS 404. The lock screen service 418 then sends a verify new PIN message to the lock setup service 420, which in turn sends a verify new PIN message to the secure OS 404. In response to verifying the new PIN, the secure OS 404 sends a success message to the lock setup service 420, which is passed back along the line to the messaging client 408. The messaging client 408 stores the new pin in the keychain in operation 716 upon receiving the success message.

[0101] Figure 8 A sequence diagram for changing settings on the safety glasses 100 according to some examples is shown. The sequence diagram shows the flow of information and calls between components and the associated state changes.

[0102] The sequence diagram begins with the messaging client 408 requesting a device security state from the lock screen service 418. This is accomplished via the glasses messaging service 430, the low power circuitry 304, and the operating system 402 on the client device 328. In this particular case, the lock screen service 418 replies to the messaging client 408 with a secure security state. The messaging client 408 then prompts the user to enter new settings, which are received by the messaging client 408 at operation 802. The messaging client 408 then retrieves the current PIN from the keychain in operation 804 and sends a set security message to the lock screen service 418. The lock screen service 418 then sends a verify PIN message to the lock settings service 420, which in turn passes the verify PIN message to the secure OS 404.

[0103] The security OS 404 then replies to the lock setup service 420 with a success message or a failure message, which is passed back along the line to the messaging client 408. In the case of a failure message, the messaging client 408 presents an appropriate failure dialog box or animation on the client device 328.

[0104] In the case of a success message, the new setting is stored in the lock screen service 418 in operation 806. The new setting may, for example, relate to the amount of time since the glasses 100 were taken off, or the loss of proximity between the glasses 100 and the client device 328 to which the glasses are locked. Additionally, if the new setting requires or is specified by the new setting, the lock screen service 418 may set an alert with the alert management service 422 in operation 810. Finally, upon receiving the success message, in operation 808, the messaging client 408 updates the UI with any settings that define changes to the UI.

[0105] Fig. 9 A sequence diagram for clearing a PIN on safety glasses 100 is shown according to some examples. The sequence diagram shows the flow of information and calls between components and associated state changes.

[0106] The sequence diagram begins with the messaging client 408 requesting the device security state from the lock screen service 418. In this particular case, the lock screen service 418 replies to the messaging client 408 with a secure security state. The messaging client 408 then retrieves the current PIN from the keychain in operation 902 and sends a set security state message to the lock screen service 418 that includes both the PIN and a clear PIN instruction. The lock screen service 418 then sends a clear lock message to the lock setup service 420. The lock setup service 420 sends a remove password message to the secure OS 404, which verifies the PIN and, if verified, clears the PIN from the key master 406.

[0107] The lock screen service 418 then sends a Verify No PIN message to the lock settings service 420. In response to the success message from the lock settings service 420, the lock screen service 418 sends a Set Lock Status message to the lock settings service 420 to set the lock status to false. The lock screen service 418 then clears all settings in operation 906 and clears any alarms in operation 904.

[0108] The lock screen service 418 then passes a success message to the messaging client 408 , which removes the PIN from the keychain in operation 908 .

[0109] Fig. 10A and Fig. 10B A sequence diagram is shown for pairing a client device 328 with glasses 100 according to some examples. The sequence diagram shows the flow of information and calls between components and the associated state changes.

[0110] Fig. 10AThe sequence is shown for the case of safety glasses 100. The messaging client 408 sends a pairing request to the low power circuit system 304 via the Bluetooth controller 434 and the glasses messaging service 430. The pairing request is passed to the lock screen service 418, which asks the lock screen service 418 from the lock setting service 420 whether the glasses 100 are secure. In this case, the lock setting service 420 responds with true, indicating that the glasses are secure. The lock screen service 418 then sends a get lock status message to the lock setting service 420 to determine whether the glasses are locked.

[0111] If the lock settings service 420 responds with true, indicating that the device is locked, the lock screen service 418 responds to the low power circuitry 304 with a not allowed message, and pairing does not occur. The low power circuitry 304 in turn provides a failure message to the messaging client 408.

[0112] If the lock settings service 420 responds with false, indicating that the device is not locked, the lock screen service 418 responds to the low power circuitry 304 with an allow message, and pairing occurs. The low power circuitry 304 in turn provides a success message to the messaging client 408.

[0113] In operation 1002 , if there is no response to the pairing request from the lock screen service 418 within the timeout duration, the low power circuitry 304 does not allow pairing.

[0114] Fig. 10B The sequence is shown in the case where the glasses 100 are not secure. The messaging client 408 sends a pairing request to the low-power circuit system 304 via the Bluetooth controller 434 and the glasses messaging service 430. The pairing request is passed to the lock screen service 418, which asks the lock screen service 418 from the lock setting service 420 whether the glasses 100 are secure. In this case, the lock setting service 420 responds with false, indicating that the glasses are not secure. The lock screen service 418 responds to the low-power circuit system 304 with an allow message, and pairing occurs. The low-power circuit system 304 in turn provides a success message to the messaging client 408.

[0115] In operation 1004 , if there is no response to the pairing request from the lock screen service 418 within the timeout duration, the low power circuitry 304 does not allow pairing.

[0116] Fig.11A and Fig. 11B A sequence diagram is shown relating to security and boot states and associated authorizations of glasses 100 according to some examples. The sequence diagram shows the flow of information and calls between components and associated state changes.

[0117] exist Fig.11AIn operation 1102, the messaging client 408 determines that the feature to be executed can be blocked by the lock screen, that is, the feature is not allowed to run when the glasses 100 are in a secure and locked state. The messaging client 408 then sends a get device security message to the lock screen service 418. The lock screen service 418 responds with a message including the security status and boot status of the glasses 100.

[0118] In operation 1104, if the lock status is true, an unlock dialog is displayed on either or both of the client device 328 and the glasses 100. In operation 1104, if the lock status is false, the messaging client 408 proceeds with the feature normally in operation 1106.

[0119] exist Fig. 11B In operation 1108, the messaging client 408 determines that the feature does not work in direct boot. The messaging client 408 sends a get device security message to the lock screen service 418. The lock screen service 418 responds with a message including the security status and boot status of the glasses 100.

[0120] In operation 1104, if the direct boot status is true and the feature does not work in direct boot, then in operation 1110, an unlock dialog is displayed on either or both of the client device 328 and the glasses 100. In operation 1104, if the direct boot status is false, then the messaging client 408 proceeds with the feature normally in operation 1112.

[0121] Fig.12 A sequence diagram for PIN entry on safety glasses 100 according to some examples is shown. The sequence diagram shows the flow of information and calls between components and associated state changes.

[0122] The sequence begins with an unlock dialog box being displayed on either or both of the client device 328 and the glasses 100 in operation 1202. User input for unlocking the glasses 100 is received in operation 1204. The messaging client 408 retrieves the PIN from the keychain and sends a Verify Pin message to the lock screen service 418. If the lock screen service 418 responds to the messaging client 408 with a success message, the messaging client 408 proceeds normally in operation 1206.

[0123] If the lock screen service 418 responds to the messaging client 408 with a failure message and the number of attempts has exceeded the specified amount, then in operation 1208, the relevant lock information is displayed on either or both of the client device 328 and the glasses 100. If the lock screen service 418 responds to the messaging client 408 with a failure message and the number of attempts has not exceeded the specified amount, then in operation 1210, a user prompt is displayed on either or both of the client device 328 and the glasses 100 to allow the user to manually enter a PIN.

[0124] Fig.13 A lock screen UI 1300 for manually entering a PIN is shown for use on glasses 100 in some examples. Glasses 100 wake up upon receiving a touch input on touchpad 124 or a press on one of buttons 126. In response and in some examples, if glasses 100 are locked, lock screen UI 1300 is presented, prompting for entry of a PIN. As can be seen, lock screen UI 1300 includes a keyboard display 1302 and an input field 1304.

[0125] As shown by the tap and swipe input 1306, the keyboard display 1302 is traversed by forward and backward swipe inputs received on the touchpad 124, and upon receiving a tap input on the touchpad 124, the centrally located highlighted number may be selected for inclusion and display in the input field 1304. Upon receiving a correct PIN, the display on the glasses 100 transforms to a user interface screen corresponding to the AR functionality of the glasses 100.

[0126] Fig.14 1400 is a flowchart 1400 illustrating operations performed by the glasses 100 and the client device 328 according to some examples. For purposes of illustration, the operations of the flowchart 1400 are described herein as occurring serially or linearly. However, multiple operations of the flowchart 1400 may occur in parallel. Additionally, the operations of the flowchart 1400 need not be performed in the order shown and / or one or more blocks in the flowchart 1400 need not be performed and / or may be replaced by other operations.

[0127] The method begins at operation 1402, where the glasses 100, which are in an off state or a sleep state to save power, receive an activation input in operation 1402. This may be, for example, the pressing of a button 126 upon initial power-up or in a sleep state.

[0128] In response to receiving the activation input, in operation 1404, the glasses 100 execute the direct boot routine discussed above. In direct boot mode, only certain direct boot aware applications can and are allowed to execute. Direct boot mode is indicated by the LOCKED_BOOT_COMPLETED system status and system status message, and triggers the execution of direct boot aware applications. Optionally, in operation 1406, a startup UI or boot screen is displayed by the glasses 100.

[0129] In operation 1408, the glasses 100 receive an instruction to perform a function. This may be sent from the client device 328, or may be a function indicated by a user input on the glasses 100, such as pressing a button to capture an image or video.

[0130] In operation 1410, the glasses 100 or the client device 328 determines whether the function requires the glasses 100 to be unlocked. If so, the method proceeds to operation 1412. If not, in operation 1426, the glasses 100 or the client device 328 determines whether the function is compatible with direct boot. If not, the method proceeds to operation 1412. If the function is direct boot compatible, the function is performed at operation 1428, and the method returns to operation 1408 and proceeds further therefrom.

[0131] If (as determined in operation 1410 or operation 1426) the function requires unlocking or is incompatible with direct boot, then in operation 1412 the glasses display a lock screen UI to the user. In operation 1414 the glasses 100 receive and verify the PIN. This can be from a keychain on the client device 328 (in response to an "unlock" user input), a manual user input of a PIN on the glasses 100 100, or a manual input on the client device 328. If the PIN is not verified, then the unlock fails and the glasses 100 display an error message or animation in operation 1424. The method then returns to operation 1412 for manual PIN entry. Multiple failed attempts will be handled as discussed above and are not shown here for clarity.

[0132] If the PIN is verified in operation 1416, the glasses 100 complete the full boot process in operation 1418 and execute the function in operation 1420. Then, in operation 1422, operation of the glasses continues normally.

[0133] Examples of locked or unlocked functions that are compatible with direct boot are management functions, such as setting the display brightness of the glasses or the volume of built-in or linked speakers (e.g., wireless headphones) from the messaging client 408. Examples of locked or unlocked functions that are not compatible with direct boot are higher-level functions that do not involve the privacy of the user or the device itself (e.g., capturing photos or videos, checking for firmware updates, or performing firmware updates). Examples of functions that are not compatible with direct boot and are locked (i.e., only available in the unlocked state) include: selecting augmented reality effects to apply to the video feed from the camera device ("Opening a LENS") and sending, posting, or uploading captured images or recorded videos.

[0134] Networked computing environment

[0135] Fig.15 is a block diagram illustrating an example messaging system 1500 for exchanging data (e.g., messages and associated content) over a network. The messaging system 1500 includes multiple instances of a client device 328, each of which hosts a number of applications including a messaging client 408 and other applications 1502. Each messaging client 408 is communicatively coupled to other instances of the messaging client 408 (e.g., hosted on respective other client devices 328), a messaging server system 1504, and a third-party server 1506 via a network 330 (e.g., the Internet). The messaging client 408 can also communicate with the locally hosted application 1502 using an application program interface (API).

[0136] The messaging clients 408 are able to communicate and exchange data with other messaging clients 408 and with the messaging server system 1504 via the network 330. The data exchanged between the messaging clients 408 and between the messaging clients 408 and the messaging server system 1504 includes functions (e.g., commands for activating functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0137] The messaging server system 1504 provides server-side functionality to certain messaging clients 408 via the network 330. Although certain functionality of the messaging system 1500 is described herein as being performed by the messaging client 408 or by the messaging server system 1504, the location of certain functionality within the messaging client 408 or within the messaging server system 1504 may be a design choice. For example, it may be technically preferable to initially deploy certain technologies and functionality within the messaging server system 1504, but to later migrate the technologies and functionality to a messaging client 408 where the client device 328 has sufficient processing power.

[0138] The messaging server system 1504 supports various services and operations provided to the messaging clients 408. Such operations include transmitting data to the messaging clients 408, receiving data from the messaging clients 408, and processing data generated by the messaging clients 408. As examples, the data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. The data exchange within the messaging system 1500 is activated and controlled by functions available via the user interface (UI) of the messaging client 408.

[0139] Turning now specifically to the messaging server system 1504, an application program interface (API) server 1508 is coupled to and provides a programming interface to an application server 1512. The application server 1512 is communicatively coupled to a database server 1514, which facilitates access to a database 1518 that stores data associated with messages processed by the application server 1512. Similarly, a web server 1522 is coupled to and provides a web-based interface to the application server 1512. To this end, the web server 1522 handles incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0140] The application program interface (API) server 1508 receives and transmits message data (e.g., commands and message payloads) between the client device 328 and the application server 1512. Specifically, the application program interface (API) server 1508 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the messaging client 408 to activate the functions of the application server 1512. The application program interface (API) server 1508 exposes various functions supported by the application server 1512, including: account registration; login functionality; sending messages from a particular messaging client 408 to another messaging client 408 via the application server 1512, sending media files (e.g., images or videos) from the messaging client 408 to the messaging server 1510, and possible access by another messaging client 408; setting up a collection of media data (e.g., a story); retrieving a friend list of a user of the client device 328; retrieving such a collection; retrieving messages and content; adding and removing entities (e.g., friends) to an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to the messaging client 408).

[0141] The application server 1512 hosts several server applications and subsystems, including, for example, a messaging server 1510, an image processing server 1516, and a social network server 1520. The messaging server 1510 implements several message processing techniques and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 408. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to the messaging client 408. In view of the hardware requirements for such processing, other processor- and memory-intensive processing of data can also be performed on the server side by the messaging server 1510.

[0142] The application server 1512 also includes an image processing server 1516 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 1510.

[0143] The social network server 1520 supports various social networking functions and services and makes these functions and services available to the messaging server 1510. To this end, the social network server 1520 maintains and accesses an entity graph within the database 1518. Examples of functions and services supported by the social network server 1520 include identifying other users with whom a particular user in the messaging system 1500 has relationships or that the particular user is "following", and also includes identifying interests and other entities of a particular user.

[0144] Returning to the messaging client 408, the features and functionality of the external resource (e.g., application 1502 or applet) are available to the user via the interface of the messaging client 408. In this context, "external" refers to the fact that the application 1502 or applet is external to the messaging client 408. External resources are typically provided by a third party, but may also be provided by the creator or provider of the messaging client 408. The messaging client 408 receives a user's selection of an option for launching or accessing features of such an external resource. The external resource may be an application 1502 installed on the client device 328 (e.g., a "local application"), or a small-scale version of an application hosted on the client device 328 or remote from the client device 328 (e.g., on a third-party server 1506) (e.g., a "applet"). The small-scale version of the application includes a subset of the features and functionality of the application (e.g., a full-scale native version of the application) and is implemented using a markup language document. In some examples, a small-scale version of an application (e.g., a "mini program") is a web-based markup language version of the application and is embedded in the messaging client 408. In addition to using markup language documents (e.g., .*ml files), mini programs can also contain scripting languages ​​(e.g., .*js files or .json files) and style sheets (e.g., .*ss files).

[0145] In response to receiving the user's selection of the option for starting or accessing the feature of the external resource, the messaging client 408 determines whether the selected external resource is a web-based external resource or a locally installed application 1502. In some cases, the application 1502 locally installed on the client device 328 can be independent of the messaging client 408 and separately started with the messaging client 408, such as by selecting an icon corresponding to the application 1502 on the main screen of the client device 328 to start like this. As used herein, the icon can include one or both of text and graphic elements. The small-scale version of such an application can be started or accessed via the messaging client 408, and in some examples, any part of the small-scale application cannot be accessed outside the messaging client 408 or a limited part of the small-scale application can be accessed. The message client 408 can receive a markup language document associated with the small-scale application from, for example, a third-party server 1506 and process such a document to start the small-scale application.

[0146] In response to determining that the external resource is a locally installed application 1502, the messaging client 408 instructs the client device 328 to launch the external resource by executing the locally stored code corresponding to the external resource. In response to determining that the external resource is a web-based resource, the messaging client 408 communicates with (for example) a third-party server 1506 to obtain a markup language document corresponding to the selected external resource. The messaging client 408 then processes the obtained markup language document to present the web-based external resource within the user interface of the messaging client 408.

[0147] The messaging client 408 can notify the user of the client device 328 or other users (e.g., "friends") associated with such a user of activities occurring in one or more external resources. For example, the messaging client 408 can provide a notification related to the current or recent use of an external resource by one or more members in a user group to a participant in a conversation (e.g., a chat session) in the messaging client 408. One or more users can be invited to join an active external resource or to start an external resource that is recently used but currently inactive (in a friend group). The external resource can provide the participants of a conversation using the corresponding messaging client 408 with the ability to share items, states, conditions, or locations in an external resource with one or more members in a user group to a chat session. The shared item can be an interactive chat card, and the members of the chat can use it to interact, for example, to start the corresponding external resource, view specific information in the external resource, or bring the members of the chat to a specific location or condition in the external resource. In a given external resource, a response message can be sent to the user on the messaging client 408. Based on the current context of the external resource, the external resource can selectively include different media items in the response.

[0148] Fig.16 1600 is a block diagram illustrating a software architecture 1604 that may be installed on any one or more of the devices described herein. The software architecture 1604 is supported by hardware, such as a machine 1602, including a processor 1620, a memory 1626, and an I / O component 1638. In this example, the software architecture 1604 may be conceptualized as a stack of layers, in which each layer provides specific functionality. The software architecture 1604 includes layers such as an operating system 1612, a library 1608, a framework 1610, and an application 1606. In operation, the application 1606 invokes an API call 1650 through the software stack and receives a message 1652 in response to the API call 1650.

[0149] The operating system 1612 manages hardware resources and provides public services. The operating system 1612 includes, for example, a kernel 1614, services 1616, and drivers 1622. The kernel 1614 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 1614 provides memory management, processor management (e.g., scheduling), component management, networking and security settings, and other functions. Services 1616 can provide other public services to other software layers. Drivers 1622 are responsible for controlling or interfacing with the underlying hardware. For example, drivers 1622 may include display drivers, camera drivers, or Low-power drivers, Flash drivers, Serial communications drivers (e.g., Universal Serial Bus (USB) drivers), Drivers, audio drivers, power management drivers, etc.

[0150] The libraries 1608 provide a low-level common infrastructure used by the applications 1606. The libraries 1608 may include a system library 1618 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the libraries 1608 may include an API library 1624, such as a media library (e.g., a library for supporting presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for presentation in two dimensions (2D) and three dimensions (3D) in graphics content on a display), a database library (e.g., SQLite providing various relational database functions), a web library (e.g., WebKit providing web browsing functions), etc. The library 1608 may also include various other libraries 1628 to provide many other APIs to the application 1606 .

[0151] The framework 1610 provides a high-level common infrastructure used by the applications 1606. For example, the framework 1610 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. The framework 1610 can provide a wide range of other APIs that can be used by the applications 1606, some of which can be specific to a particular operating system or platform.

[0152] In an example, applications 1606 may include a home application 1636, a contacts application 1630, a browser application 1632, a book reader application 1634, a location application 1642, a media application 1644, a messaging application 1646, a game application 1648, and a variety of other applications such as third-party applications 1640. Applications 1606 are programs that execute functions defined in the program. Various programming languages ​​may be used to create one or more of the applications 1606 constructed in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C language or assembly language). In a specific example, third-party applications 1640 (e.g., those created by an entity other than the vendor of a particular platform using ANDROID TM or IOS TM Software Development Kit (SDK) can be used to develop applications on platforms such as IOS TM ANDROID TM , Mobile software running on the mobile operating system of the phone or another mobile operating system. In this example, the third-party application 1640 can call the API call 1650 provided by the operating system 1612 to facilitate the functions described in this article.

[0153] Fig.171700 in which instructions 1710 (e.g., software, programs, applications, applet, app, or other executable code) may be executed to cause the machine 1700 to perform any one or more of the methods discussed herein. For example, the instructions 1710 may cause the machine 1700 to perform any one or more of the methods described herein. The instructions 1710 transform a general-purpose, unprogrammed machine 1700 into a specific machine 1700 that is programmed to perform the functions described and illustrated in the manner described. The machine 1700 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1700 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1700 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular phone, a smart phone, a mobile device, a head-mounted device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1710 that specify actions to be taken by the machine 1700, either sequentially or otherwise. In addition, while only a single machine 1700 is shown, the term "machine" shall also be taken to include a collection of machines that individually or jointly execute instructions 1710 to perform any one or more of the methodologies discussed herein.

[0154] The machine 1700 may include a processor 1702, a memory 1704, and an I / O component 1706, which may be configured to communicate with each other via a bus 1744. In an example, the processor 1702 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1708 that executes instructions 1710 and a processor 1712. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Fig.17 Multiple processors 1702 are shown, but the machine 1700 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0155] The memory 1704 includes a main memory 1714, a static memory 1716, and a storage unit 1718, which are all accessible by the processor 1702 via the bus 1744. The main memory 1704, the static memory 1716, and the storage unit 1718 store instructions 1710 that implement any one or more of the methods or functions described herein. During execution of the instructions 1710 by the networked system 300, the instructions 1710 may also reside, in whole or in part, within the main memory 1714, within the static memory 1716, within the machine-readable medium 1720, within the storage unit 1718, within at least one of the processors 1702 (e.g., within a cache memory of the processor), or within any suitable combination thereof.

[0156] I / O components 1706 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 1706 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It is to be understood that I / O components 1706 may include Fig.17 1706. In various examples, the I / O components 1706 may include output components 1728 and input components 1732. The output components 1728 may include visual components (e.g., displays such as plasma display panels (PDPs), light emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. The input components 1732 may include alphanumeric input components (e.g., keyboards, touch screens configured to receive alphanumeric input, optical keyboards, or other alphanumeric input components), pointing-based input components (e.g., mice, touch pads, trackballs, joysticks, motion sensors, or other pointing instruments), tactile input components (e.g., physical buttons, touch screens that provide location and / or force of touch or touch gestures, or other tactile input components), audio input components (e.g., microphones), etc.

[0157] In another example, the I / O component 1706 may include: a biometric component 1734, a motion component 1736, an environmental component 1738, or a positioning component 1740, as well as various other components. For example, the biometric component 1734 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1736 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environment component 1738 includes, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The positioning component 1740 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure that can obtain altitude), an orientation sensor component (e.g., a magnetometer), etc.

[0158] A variety of technologies may be used to achieve communication. I / O components 1706 also include communication components 1742 that are operable to couple networked system 300 to network 1722 or device 1724 via coupling 1730 and coupling 1726, respectively. For example, communication components 1742 may include a network interface component or another suitable device that interfaces with network 1722. In other examples, communication components 1742 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Components (e.g. Low power consumption), Device 1724 may be another machine or any of a variety of peripheral devices (eg, a peripheral device coupled via USB).

[0159] In addition, the communication component 1742 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 1742 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting the following: a one-dimensional barcode, such as a universal product code (UPC) barcode; a multi-dimensional barcode, such as a Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Hypercode, UCC RSS-2D barcode, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 1742, such as a location via Internet Protocol (IP) geolocation, a location via Internet Protocol (IP), ... The location of signal triangulation, the location of an NFC beacon signal via detection that can indicate a specific location, etc.

[0160] Various memories (e.g., memory 1704, main memory 1714, static memory 1716, and / or memory of processor 1702) and / or storage unit 1718 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1710) when executed by processor 1702 cause various operations to implement the disclosed examples.

[0161] The instructions 1710 may be transmitted or received via a network interface device (e.g., a network interface component included in the communication component 1742) using a transmission medium and using any of a number of well-known transmission protocols (e.g., the Hypertext Transfer Protocol (HTTP)) over the network 1722. Similarly, the instructions 1710 may be transmitted or received to the device 1724 via a coupling 1726 (e.g., a peer-to-peer coupling) using a transmission medium.

[0162] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions. Instructions may be transmitted or received over a network using a transmission medium via a network interface device.

[0163] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.

[0164] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, another type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.

[0165] "Component" refers to a device, physical entity or logic with boundaries defined by function or subroutine calls, branch points, APIs or other technologies provided for partitioning or modularizing specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be encapsulated functional hardware units designed for use with other components, and are usually part of a program that performs a specific function of related functions. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., independent computer systems, client computer systems or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component may also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It will be understood that the decision to mechanically implement the hardware component in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., configured by software) circuit system can be driven by cost and time considerations. Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an example where a hardware component is temporarily configured (e.g., programmed), each of the hardware components does not need to be configured or instantiated at any time. For example, where a hardware component includes a general purpose processor that is configured by software to be a special purpose processor, the general purpose processor may be configured as different special purpose processors (e.g., including different hardware components) at different times. Software configures one or more specific processors accordingly, such as to constitute a specific hardware component at one time and to constitute different hardware components at different times. Hardware components may provide information to other hardware components and receive information from other hardware components. Thus, the described hardware components may be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between two or more hardware components or among two or more hardware components. In the example where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessible to multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device coupled to it in communication. Then, other hardware components can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., the collection of information). The various operations of the example methods described herein can be performed at least in part by a temporary configuration (e.g., by software) or a permanent configuration to perform one or more processors of the related operations. Whether it is a temporary configuration or a permanent configuration, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operation of the method can be performed by one or more processors or the parts implemented by the processor. In addition, one or more processors can also operate to support the execution of the related operations in the "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some of the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (for example, the Internet) and via one or more appropriate interfaces (for example, API). The execution of some operations in the operation can be distributed between processors, not only resides in a single machine, but also deployed across multiple machines. In some examples, a processor or a part implemented by a processor can be located in a single geographical location (for example, in a home environment, an office environment or a server cluster). In some examples, a processor or a part implemented by a processor can be distributed across multiple geographical locations.

[0166] "Computer-readable media" refers to both machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium" and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.

[0167] "Ephemeral messages" are messages that can be accessed for a limited duration of time. Ephemeral messages can be text, images, videos, etc. The access time of an ephemeral message can be set by the sender of the message. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is ephemeral.

[0168] "Machine storage medium" refers to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions, routines, and / or data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" explicitly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal media".

[0169] A "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and produces corresponding output signals that are applied to operate a machine. For example, a processor may be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.

[0170] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions executed by a machine, and "signal medium" includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0171] Changes and modifications may be made to the disclosed examples without departing from the scope of the present disclosure. These and other changes or modifications are intended to be included within the scope of the present disclosure as expressed in the claims.

Claims

1. A method, performed by one or more processors in a head mounted device including one or more display devices, the method include: receiving activation input; performing a partial startup of the head mounted device; receiving instructions for performing functions; determining whether the instruction is permitted for partially initiated execution; Based on the instruction being permitted for partially initiated execution, executing the instruction; as well as Based on the instruction not being allowed to be executed for partial startup, booting of the head mounted device is completed.

2. The method according to claim 1, further comprising: include: determining whether the instruction requires user authentication to be executed; as well as Based on the instructions being partially launch compliant and not requiring user authentication, the instructions are executed.

3. The method according to claim 1, in, The partial startup of the head mounted device includes starting a lock screen service and a camera device service.

4. The method according to claim 1, further comprising: include: determining whether the instruction requires user authentication to be executed; as well as Based on the instruction requiring user authentication, a user authentication process is performed regardless of whether the instruction is partially launch compatible.

5. The method according to claim 4, further comprising: include: Verify correct user authentication; Completing the startup of the head mounted device; as well as Execute the instructions.

6. The method according to claim 5, in, Verifying proper user authentication includes: Receive a password from the associated mobile device.

7. The method according to claim 6, further comprising: include: Based on proper user authorization not being verified using the password received from the associated mobile device, the user is prompted via the head mounted device to enter a password.

8. The method according to claim 6, further comprising: include: An authentication state of the head mounted device is maintained based on a proximity of the head mounted device to the associated mobile device.

9. A head mounted device, include: one or more camera devices; one or more display devices; one or more processors; as well as a memory storing instructions that, when executed by the one or more processors, configure the device to perform operations comprising: receiving activation input; performing a partial startup of the head mounted device; receiving instructions for performing functions; determining whether the instruction is permitted for partially initiated execution; Based on the instruction being allowed for partially enabled execution, executing the instruction; and Based on the instruction not being allowed to be executed for partial startup, booting of the head mounted device is completed.

10. The head mounted device according to claim 9, in, The operations also include: determining whether the instruction requires user authentication to be executed; and Based on the instructions being partially launch compliant and not requiring user authentication, the instructions are executed.

11. The head mounted device according to claim 9, in, The operations also include: determining whether the instruction requires user authentication to be executed; and Based on the instruction requiring user authentication, a user authentication process is performed regardless of whether the instruction is partially launch compatible.

12. The head mounted device according to claim 11, in, The operations also include: Verify correct user authentication; completing the startup of the head mounted device; and Execute the instructions.

13. The head mounted device according to claim 12, in, Verifying proper user authentication includes: Receive a password from the associated mobile device.

14. The head mounted device according to claim 13, in, The operations also include: Based on proper user authorization not being verified using the password received from the associated mobile device, the user is prompted via the head mounted device to enter a password.

15. The head mounted device according to claim 13, in, The operations also include: An authentication state of the head mounted device is maintained based on a proximity of the head mounted device to the associated mobile device.

16. A non-transitory computer-readable storage medium comprising instructions that, when executed by a head-mounted device comprising one or more display devices and one or more camera devices, cause the head-mounted device to perform an operation, the operation include: receiving activation input; performing a partial startup of the head mounted device; receiving instructions for performing functions; determining whether the instruction is permitted for partially initiated execution; Based on the instruction being permitted for partially initiated execution, executing the instruction; as well as Based on the instruction not being allowed to be executed for partial startup, booting of the head mounted device is completed.

17. The non-transitory computer readable storage medium of claim 16, in, The operations also include: determining whether the instruction requires user authentication to be executed; and Based on the instructions being partially launch compliant and not requiring user authentication, the instructions are executed.

18. The non-transitory computer readable storage medium of claim 16, in, The partial startup of the head mounted device includes starting a lock screen service and a camera device service.

19. The non-transitory computer readable storage medium of claim 16, in, The operations also include: determining whether the instruction requires user authentication to be executed; and Based on the instruction requiring user authentication, a user authentication process is performed regardless of whether the instruction is partially launch compatible.

20. The non-transitory computer readable storage medium of claim 19, in, The operations also include: Verify correct user authentication; completing the startup of the head mounted device; and Execute the instructions.

Images