How to unlock an electronic device

Abstract

To provide a security function such as software locking of a head-mounted display (HMD).SOLUTION: A method of unlocking software includes: detecting an input gesture operable to initiate password entry when an electronic device 10 is in a software locked state; randomly assigning a predefined set of input gestures to a predefined set of password symbols; detecting one of the assigned input gestures via the gesture input device; comparing the entered password symbols corresponding to the detected input gestures to a stored unlock password via the processing unit; and when the entered password symbols match the stored unlock password, enabling the operation of the electronic device in a software unlocked state.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates generally to unlocking software-locked electronic devices, and more specifically, to unlocking electronic devices operable to identify gesture sequences on a touch-sensitive device and / or camera. [Background technology]

[0002] With the rapid advancement of technology, manufacturers of mobile devices (e.g., head mounted displays, smartphones) are constantly striving to add greater functionality in smaller form factors with user interface capabilities to realize the convenience of mobility. For example, in addition to already advanced data capabilities, mobile electronic devices must include software lock and unlock capabilities for privacy and data protection.

[0003] Head-mounted displays (HMDs) are being developed for a variety of applications including military, commercial, industrial, firefighting, entertainment, etc. In many of these applications, providing security features such as a software lock on the HMD is particularly valuable because HMDs are becoming increasingly lightweight and are not fixed to a physical location, making them easily taken away or used by unauthorized individuals.

[0004] The goal of greater functionality may be at odds with unlocking an electronic device, which requires an additional action by the user, such as entering a password. Speaking the password within listening distance of another person or a recording device does not provide reliable security. Entering a password is not applicable to electronic devices that do not have a keyboard input device. Similarly, an electronic device may not have other input means available, such as a fingerprint or biometric scanner, or a camera with facial recognition capabilities. In such situations, a touchpad or touchscreen may be the preferred electronic input device. Thus, there is a need for a convenient and secure method to unlock software-locked electronic devices. Summary of the Invention

[0005] The present disclosure provides a software unlocking method for unlocking functionality of an electronic device, and the electronic device produced thereby.

[0006] In a first exemplary embodiment, a software unlocking method includes providing an electronic device with a processing unit and a gesture input device connected to the processing unit. The method further includes detecting an input gesture operable to initiate a password input in a situation where the electronic device is in a software locked state, and randomly assigning a set of predefined input gestures to a set of predefined password symbols. The software unlocking method further includes detecting one of the assigned input gestures via the gesture input device, and comparing an input password symbol corresponding to the detected input gesture with a stored unlock password via the processing unit. Furthermore, if the input password symbol matches the stored unlock password, enabling the electronic device to operate in a software unlocked state.

[0007] In a second exemplary embodiment, an electronic device includes a processing unit and a gesture input device connected to the processing unit. The electronic device further includes a projector connected to the processing unit and operable to generate an image-bearing light, and the planar waveguide assembly operable to couple to the image-bearing light. The planar waveguide assembly is operable to generate a virtual image display. Furthermore, the gesture input device is operable to detect an input gesture for initiating a password input in a software-locked state, and the processing unit is operable to randomly assign a set of predefined input gestures to a set of predefined password symbols. The virtual image display is operable to display a password input screen showing the randomly assigned set of input gestures corresponding to the set of password symbols. [Brief description of the drawings]

[0008] The accompanying drawings, which are incorporated as part of the specification, illustrate embodiments of the presently disclosed subject matter and illustrate selected principles and teachings of the present disclosure. However, the drawings do not illustrate every possible implementation of the presently disclosed subject matter, and are not intended to limit the scope of the present disclosure in any way.

[0009] [Figure 1A] 1 illustrates an electronic device according to one embodiment of the present disclosure.

[0010] [Figure 1B] 1 illustrates an electronic device according to one embodiment of the present disclosure.

[0011] [Diagram 2] 1 illustrates a portion of a software lock screen display according to one embodiment of the present disclosure.

[0012] [Diagram 3]13 illustrates a portion of a password entry screen according to one embodiment of the present disclosure.

[0013] [Figure 4] 13 illustrates a portion of a password entry screen according to one embodiment of the present disclosure.

[0014] [Diagram 5] 1 is a flow chart of a software unlocking method according to one embodiment of the present disclosure.

[0015] [Figure 6] 1 is a flow chart of a software unlocking method according to one embodiment of the present disclosure.

[0016] [Figure 7] 1 is a flow chart of a software unlocking method according to one embodiment of the present disclosure.

[0017] [Figure 8] 1 is a flow chart of a software unlocking method according to one embodiment of the present disclosure.

[0018] [Figure 9] 1 is a flow chart of a software unlocking method according to one embodiment of the present disclosure.

[0019] [Figure 10] 1 is a flowchart of a software unlocking method according to an embodiment of the present disclosure.

[0020] [Figure 11] 13 illustrates a portion of a password entry screen according to one embodiment of the present disclosure.

[0021] [Figure 12] 1 illustrates a general-purpose mobile computing device according to one embodiment of the present disclosure. Detailed Description

[0022] It is to be understood that the invention may assume various alternative arrangements and step sequences unless expressly specified. It is also to be understood that the specific assemblies and systems illustrated in the drawings and described in the specification are merely exemplary embodiments of the inventive concepts defined herein. Thus, specific dimensions, orientations, or other physical characteristics relating to the disclosed embodiments are not to be considered as limiting, unless expressly stated. Also, like elements in the various embodiments described herein are referred to by like reference numerals, except where otherwise noted.

[0023] In this specification, unless otherwise specified, the terms "first", "second", etc. do not necessarily indicate an order or priority relationship, but are merely used to more clearly distinguish one element or set of elements from another element or set of elements.

[0024] In the context of this disclosure, the terms "viewer," "operator," "observer," and "user" are considered equivalent and refer to a person who wears and / or uses an electronic device to view an image.

[0025] The term "set" as used herein refers to a non-empty set, as the concept of a collection of elements or members is widely understood in elementary mathematics. The term "subset" is used, unless otherwise noted, to refer to a non-empty proper subset, i.e., a subset having one or more elements of a larger set. In the case of a set S, a subset may include the complete set S. However, a "proper subset" of set S is strictly contained in set S and excludes at least one element of set S.

[0026] Head-mounted displays (HMDs) are being developed for a variety of applications, including military, commercial, industrial, firefighting, and entertainment. HMDs are operable to form a virtual color image that can be visually superimposed on a real-world image within the field of view of the HMD user. An optically transparent parallel-plate waveguide, also called a planar waveguide, transmits image-bearing light generated by a color projector system to a user of the HMD. The planar waveguide transmits the image-bearing light in a small space and directs the virtual image to the pupil of the HMD user, allowing the virtual image to be superimposed on a real-world image within the field of view of the HMD user.

[0027] In such conventional image light guides, collimated and angularly related encoded light beams from a color image projector light source are coupled into an optically transparent planar waveguide assembly by an input coupling optical element, such as an incoupling diffractive optical element. This incoupling diffractive optical element can be placed or formed on the surface of the parallel plate planar waveguide or within the waveguide. Such a diffractive optical element can be formed as, but not limited to, a diffraction grating or a holographic optical element. For example, the diffraction grating can be formed as a surface relief grating. After propagating along the planar waveguide, the diffracted color image-bearing light is directed out and back out of the planar waveguide by a similar output grating. This output grating can be configured to provide pupil expansion of the virtual image along one or more dimensions. Additionally, one or more of the output gratings can be configured to provide a holographic optical element. A diffractive rotating grating or gratings can be optically positioned along the waveguide between the input and output gratings to dilate the virtual image in one or more dimensions. The image-bearing light output from the parallel-plate planar waveguide provides an expanded eyebox to the viewer.

[0028] An optical system, such as an HMD, can generate a virtual image display. In contrast to methods of forming real images, virtual images are not formed on the display surface; i.e., if the display surface were in a position where the virtual image could be perceived, no image would be formed on that surface. Displaying virtual images has many advantages for augmented reality presentations. For example, the apparent size of the virtual image is not limited by the size or position of the display surface. Furthermore, the source object of the virtual image can be made small. For example, a magnifying glass provides a virtual image of an object. By forming a virtual image that appears to be at some distance compared to a system that projects a real image, an even more realistic viewing experience can be provided. Providing a virtual image also eliminates the need to compensate for screen artifacts, which is necessary when projecting a real image.

[0029] HMDs typically do not have a keyboard device for input. However, HMDs have a touch-sensitive pad or area on the device for gesture input. Alternatively, a camera or a combination of camera and touchpad input can be used to capture the input gestures. Providing software unlock functionality using gesture-sensitive and gesture-recognition input devices in the HMD provides a secure method of unlocking software-locked electronic devices.

[0030] A method for unlocking a software-locked electronic device allows a user to unlock the electronic device by identifying a gesture sequence that includes at least two consecutive gestures. In one embodiment, the method for unlocking a software-locked electronic device maps the identifiable gestures to symbols (letters and / or numbers, or other symbols) that are displayed to the user by a display means such as a virtual image display or a liquid crystal display (LCD), detects the sequence of gestures from an electronic touch-sensitive device and / or other gesture capture device, converts the detected sequence of gesture inputs into a symbol sequence, and compares the symbol sequence with a symbol sequence of a predefined password to determine whether to unlock the electronic device.

[0031] As shown in FIG. 1A, in one embodiment, the electronic device 10 includes an HMD in the form of glasses. The electronic device 10 includes at least a right temple arm 12 and a processing unit 14. The processing unit 14 has means for storing data, storing computer programs, and executing computer programs. Furthermore, the processing unit 14 is connected to a gesture input device 16. The gesture input device 16 is operable to transmit gestures to the processing unit 14. In one embodiment, the gesture input device 16 is a touchpad. The gesture input device 16 is operable to receive input from one or more fingers of a user 20 and / or from a stylus. Where input gestures are described as being performed by the fingers of the user 20, those skilled in the art will recognize that the input gestures may also be performed by a stylus. Gestures received by the gesture input device 16 include, but are not limited to, tapping on the gesture input device 16, swiping / dragging across the gesture input device 16. The swipes / drags may be swiped / dragged from front to back, back to front, top to bottom, bottom to top, front to back and back to front simultaneously and vice versa, bottom to top and top to bottom simultaneously and vice versa, front to back and further forward, back to front and further back, bottom to top and further down, top to bottom and further up. The gesture input device 16 may further detect whether one finger, two fingers, or three fingers are used during the gesture. The processing unit 14 may convert the sequence of gestures into a sequence of symbols. The sequence of symbols is suitable for comparison with a stored sequence of symbols to be used as a stored unlock password.

[0032] 1B, in one embodiment, the electronic device 10 further includes at least one camera 18 positioned to view hand gestures of the user 20. In one embodiment, the camera 18 is positioned adjacent to the outer periphery of the lens 22 of the eyeglasses. The field of view of the camera 18 is in front of the user 20. In another embodiment, the camera 18 is positioned on the left or right temple arm 12 of the eyeglasses, and the field of view of the camera 18 is to the side of the user 20.

[0033] The camera 18 is connected to the processing unit 14 and can be used to capture and recognize hand gestures of a user 20. The processor 14 is operable to convert the hand gestures captured by the camera 18 into a corresponding sequence of symbols that are compared to a stored unlock password.

[0034] The electronic device 10 includes a display, such as, but not limited to, a virtual image display or a physical display (e.g., a screen and / or LCD). As shown in FIG. 2, the electronic device 10 displays a lock screen 30 (virtual or physical) to the user 20 indicating that the electronic device 10 is in a software-locked state. In one embodiment, the displayed lock screen 30 includes at least a lock icon 32 indicating the software-locked state of the electronic device 10. Other icons and / or words may be displayed to inform the user 20 that the electronic device 10 is in a software-locked state. Other information may be displayed on the lock screen 30, such as, but not limited to, a date 34A, a time 34B, a charging state 34C, and a battery charge level 34D.

[0035] The electronic device 10 can be put into a sleep state in which the display is turned off, and the power level of the electronic device 10 can be conserved during periods of non-use. In one embodiment, any touch on the gesture input device 16 is a wake cue to operate the electronic device 10 in an active state. If the electronic device 10 is in a software-locked state, the wake cue causes the display to present the lock screen 30. Alternatively, a specific touch pattern on the gesture input device 16 can be used to wake up the electronic device 10. For example, the wake cue may be a two-finger tap, a single finger on the gesture input device 16 for a predetermined period of time, or a single finger swipe across the gesture input device 16. Other gestures may be defined to wake up the electronic device 10. The active state is a state of the electronic device 10 that allows input from at least the gesture input device 16 and / or the camera 18 and presents images on the display of the electronic device 10. Input from other sensors (e.g., buttons, tilt sensors, accelerometers, magnetometers, etc.) can be used to change the state of the electronic device 10 from a sleep state to an active state, and in the case of a software lock state, the lock screen 30 is displayed.

[0036] In one embodiment, the touch of the gesture input device 16 initiates the software unlocking method (see FIGS. 5-10). As shown in FIG. 3, when the user 20 initiates the software unlocking method, a password entry screen 40 (virtual or physical) is displayed to the user 20. The password entry screen 40 has a password display area 42, and when the user 20 inputs a gesture sequence, an input password symbol 48 is displayed in the password display area 42. In one embodiment, the input password symbol 48 is temporarily displayed in the password display area 42. After a predetermined period of time, the input password symbol 48 is overwritten or replaced by a masking symbol 46 to hide the input password symbol 48. The masking symbol 46 is a circle, a square, or the like, and indicates that the input password symbol 48 has been input without displaying the input password symbol 48. In this way, the user 20 can keep track of which password characters have already been input. The entered password symbol 48 also informs the user 20 which gestures the electronic device 10 recognized from the user's input gestures.

[0037] 3, in one embodiment, the password entry screen 40 also includes a gesture map area 44. In the gesture map area 44, a set of password symbols 50 is displayed with corresponding gesture symbols 52. The gesture symbols 52 indicate input gestures that can be used with the gesture input device 16 and / or the camera 18 to enter a sequence of password symbols 50. As shown in FIG. 3, in one embodiment, the complete set of password symbols 50 is the set {1, 2, 3, 4, 5, 6, 7, 8, 9, 0}. Alternatively, the password symbols 50 may be the set {A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, +}. Other sets of symbols, such as geometric shapes, may also be used as the password symbols 50.

[0038] 3, in one embodiment, a set of 56 password sub-symbols are displayed beneath the password symbol 50. The password sub-symbols 56 allow a password to be composed of a mixture of symbols from different sets of symbols 50, 56. The corresponding gestures are indicated by displaying a gesture symbol 52 next to the password symbol 50 and / or password sub-symbols 56. In one embodiment, the gesture symbols 52 and their corresponding input gestures (input gestures with the gesture input device 16 and / or camera 18) are as follows: ">" indicates a one finger swipe forward gesture, "<" indicates a one finger swipe backward gesture, "*" indicates a one finger tap gesture, and "**" indicates a two finger tap gesture. ">>" indicates a two-finger forward swipe gesture, and "< "<" indicates a two finger backward swipe gesture, "Λ" indicates a one finger bottom-up swipe gesture, "V" indicates a one finger top-down swipe gesture, "ΛΛ" indicates a two finger bottom-up swipe gesture, and "VV" indicates a two finger top-down swipe gesture. A two finger input gesture means that two fingers touch the gesture input device 16 simultaneously. Alternate gesture symbols 52 can be used to indicate password symbols 50 and password sub-symbols 56.

[0039] When an input gesture is performed, in addition to displaying the input password symbol 48 in the password display area 42, the gesture symbol 52 is highlighted in the gesture map area 44. The highlighting may include one or a combination of changing the color of the gesture symbol 52, changing the font style of the gesture symbol 52 (e.g., regular vs. italics and / or bold), underlining the gesture symbol 52, and surrounding the symbol with a box 54 or a circle. As shown in FIG. 3, in one embodiment, the box 54 highlighting the corresponding gesture symbol 52 when the input gesture is performed may include one or more dashed / dotted lines surrounding both the password symbol 50 and the secondary symbol 56 and the gesture symbol 52. As shown in FIG. 4, in one embodiment, the box 54 highlighting the corresponding gesture symbol 52 when the input gesture is performed surrounds the gesture symbol 52.

[0040] Other input gestures may be used for other functions during the execution of the software unlocking method. For example, a three-finger tap on the gesture input device 16 may be used to erase any previously entered password symbols and indicate starting the password entry sequence over again. Alternatively, the gesture may be used to erase any previously entered password symbols and indicate starting the password entry sequence over again for at least a predetermined period of time, e.g., two seconds. Placing a finger on the finger input device 16 may also indicate that any previously entered password symbols should be erased and the password entry sequence should be started over.

[0041] When camera 18 is utilized as the input gesture capture device, a "swipe" can be the movement of one or more fingers within the field of view of camera 18. For example, a forward swipe can be the movement of one or more fingers from left to right, a backward swipe can be the movement of one or more fingers from right to left, an upward swipe can be the vertical movement of one or more fingers from bottom to top, and a downward swipe can be the vertical movement of one or more fingers from top to bottom. A "tap" gesture can be the curling of one or more fingers within the field of view of camera 18.

[0042] 4, the password entry screen 70 includes randomly assigned gesture symbols 52. When the software unlocking method is initiated, the input gestures are randomly assigned to password symbols 50 and password sub-symbols 56. In this way, a person observing the user 20 and seeing the sequence of input gestures used to unlock the electronic device 10 cannot unlock the electronic device 10 by utilizing the same gesture sequence. Thus, the input gesture sequence can be made public and shown off in public without revealing the actual unlock password.

[0043] As shown in FIG. 5, in one embodiment, the software unlocking method 90 starts with the electronic device 10 in a software locked state 100. To start the software unlocking method 90, in step 102, the user 20 provides an input gesture to the gesture input device 16 and / or the camera 18. In one embodiment, the input gesture for starting the software unlocking method 90 is one or more taps. In one embodiment, the software unlocking method 90 is started by pressing a button 24 on the electronic device 10. Step 102 includes randomly assigning input gestures 52 to password symbols 50 and password sub-symbols 56. The next step 104 of the software unlocking method 90 clears any previously entered input gestures and causes the display to present the password entry screen 40. Step 106 captures the input gesture of the user 20. Step 108 converts the input gesture into an input password symbol 48 (also known as an input character). In step 110, the input password symbol 48 is compared to the stored unlock password. Step 110 is initiated by the user 20 providing an input gesture indicating that the password input sequence is complete. For example, a password input gesture for "enter" is a simultaneous swipe or pinch from the top to the middle and from the bottom to the middle. Step 112 operates based on a comparison of the input password symbol 48 to the stored unlock password. The comparison indicates that the input password symbol 48 matches or does not match the stored unlock password. If the input password symbol 48 matches the stored unlock password, the software unlock method 90 proceeds to step 116. In step 116, the electronic device 10 is unlocked and the software is enabled as shown in step 120. If the input password symbol 48 does not match the stored unlock password, the software unlock method 90 proceeds to step 114 where the input password symbol 48 is cleared.The software unlock method 90 can then return to step 104 to obtain another sequence of input gestures. Step 114 also includes randomly assigning the input gestures 52 to the password symbols 50 and secondary symbols 56.

[0044] As shown in FIG. 6, in one embodiment, the software unlock method 150 starts with the electronic device 10 in a software locked state 152. In step 154, the software unlock method 152 is started. In one embodiment, the input gesture for starting the software unlock method 150 is one or more taps. In one embodiment, the software unlock method 90 is started by pressing the button 24 of the electronic device 10. Step 156 clears all previous input gestures and displays the password entry screen 40. Step 156 also includes random assignment of the input gestures 52 to the password symbols 50 and secondary symbols 56. Step 158 captures a single input gesture. Step 160 compares this input gesture with predefined cancel gestures. If the input gesture is a cancel gesture, the software unlock method 150 returns to step 156. In one embodiment, the cancel gesture is a timeout period during which no input gesture is recognized / captured. In another embodiment, the cancel gesture is one finger touching the gesture input device 16 for a predetermined period of time, for example, two seconds. In yet another embodiment, the cancel gesture is three fingers touching the gesture input device 16.

[0045] If the input gesture is not a cancel gesture, the software unlock method 150 proceeds to step 166 where the input gesture is converted to an input password symbol 48. Step 168 appends the input password symbol 48 to the end (end) of any sequence of input password symbols 48, resulting in further accumulation. Step 174 compares the current sequence of input password symbols 48 to the stored unlock password. If the accumulated input password symbols 48 match the stored unlock password in step 176, the software unlock method 150 proceeds to step 178. In step 178, the electronic device 10 is unlocked and the software is operable in a software unlock state 180. If the accumulated input password symbols 48 do not match the stored unlock password, the software unlock method 150 proceeds to step 182. In step 182, the number of accumulated input password symbols 48 is compared to the maximum allowed number of input gestures. If the cumulative number of input password symbols 48 is the maximum allowable number of input gestures, the software unlock method 150 returns to step 156. If the cumulative number of input password symbols 48 is less than the maximum allowable number, the software unlock method 150 returns to step 158 where another input gesture is captured.

[0046] As shown in FIG. 7, in one embodiment, the software unlocking method 200 starts with the electronic device 10 in a software locked state 202. In the software locked state 202, the electronic device 10 is in a sleep state with the display turned off. Step 204 starts the software unlocking method 200 via a wake-up cue. In one embodiment, the wake-up cue is pressing a button 24 on the electronic device 10. In one embodiment, the wake-up cue is a touch on the gesture input device 16. The wake-up cue causes the display to present a lock screen 30 in step 205. In step 206, the user 20 initiates password entry via an input gesture. The input gesture may include, but is not limited to, a touch on the gesture input device 16, a virtual touch on the camera 18, or pressing a button 24 on the electronic device 10. Step 206 also includes randomly assigning the input gesture 52 to the password symbols 50 and secondary symbols 56. By starting to enter a password, the display presents the password entry screen 40 in step 207 .

[0047] In a next step 208, the input gesture is captured. In step 210, the captured input gesture is compared to a predefined cancel gesture to indicate cancellation of the password entry sequence. In one embodiment, the cancel gesture is a timeout period during which the input gesture is not recognized / captured. In another embodiment, In one embodiment, the cancel gesture is touching the gesture input device 16 with one finger for a predetermined period of time, e.g., two seconds. In yet another embodiment, the cancel gesture is touching the gesture input device 16 with three fingers. If the input gesture is a cancel gesture, the software unlock method 200 clears the previously entered password symbol 48 in step 212 and returns to step 207 to display the input password screen 40, 70 of the electronic device. Step 212 also includes random assignment of the input gesture 52 to the password symbols 50 and secondary symbols 56.

[0048] If the input gesture is not a cancel gesture, the software unlock method 200 proceeds to step 216, where the input gesture is converted to an input password symbol 48 or an input character. Step 218 appends the input password symbol 48 or character to the end of the sequence of input password symbols 48 or characters, which are then further accumulated. In step 224, the input password symbol 48 or character is displayed in the password display area 42. The input password symbol 48 or character is displayed for a predetermined, usually short, time. In step 226, the display time of the input password symbol 48 is compared to a set / predetermined period. Step 226 may compare the time that the input password symbol 48 is displayed to the set period a predetermined number of times per second. For example, step 226 may compare the time to the set period every tenth of a second. If the time that the input password symbol 48 is displayed does not match or exceed the set period, the software unlock method 200 returns to step 224. If the time during which the input password symbol 48 is displayed exceeds the set time, the software unlocking method 200 executes step 230, where the masking symbol 46 is displayed in place of the input password symbol 48 or the character corresponding to the input gesture. In this way, the input password symbol 48 or the input password character is removed from the password display area 42.

[0049] Next, in step 232, the sequence of input password symbols 48 entered so far is compared to the stored unlock password. If, in step 234, the accumulated input password symbols 48 match the stored unlock password, the software unlock method 200 proceeds to step 236. In step 236, the electronic device 10 is unlocked and the software is operable in a software unlocked state 238. In one embodiment, the step 230 of displaying the masking symbol 46 after a set period of time is performed simultaneously with the step 232 of comparing the sequence of input password symbols 48 to the stored unlock password. In other words, in one embodiment, the timer of step 230 operates independently of step 232.

[0050] If the sequence of input password symbols 48 does not match the stored unlock password, the software unlock method 200 proceeds to step 240. In step 240, the number of accumulated input password symbols 48 is compared to the maximum allowable number of input gestures and / or input password symbols 48. If the accumulated number of input password symbols 48 is the maximum allowable number, the software unlock method 200 clears the entered password symbols 48 in step 244 and returns to step 207. If the accumulated number of input password symbols 48 is less than the maximum allowable number, the software unlock method 200 returns to step 208 where another input gesture is captured.

[0051] As shown in FIG. 8, in one embodiment, the software unlocking method 300 starts with the electronic device 10 in a software locked state 302. In the software locked state 302, the electronic device 10 is in a sleep state and the display is turned off. Step 304 starts the software unlocking method 300 via an input gesture (such as one or more taps on the gesture input device 16) or pressing a button 24 on the electronic device. Step 304 allocates computer memory or sets computer variables, etc., to initial values. Step 305 determines whether the electronic device 10 is in a sleep state. If the electronic device 10 is in a sleep state, the display presents the lock screen 30 in step 306. In step 308, the user 20 initiates a password entry via an input gesture, such as, but not limited to, a touch on the gesture input device 16 or a virtual touch on the camera 18. Upon initiating password entry, in step 310, the software unlocking method 300 randomly assigns an input gesture and its corresponding corresponding gesture symbol 52 to a password symbol 50, 56. If in step 305, the software unlocking method 300 determines that the electronic device 10 is not in a sleep state, the software unlocking method 300 proceeds to step 310.

[0052] Step 310 also clears any previously entered password symbols 48 (from a previous attempt to enter a stored unlock password) and / or input gestures. In a next step 312, the display presents the password entry screen 40. In a next step 314, the input gesture is captured. In step 316, the captured input gesture is compared to a predefined cancel gesture used to indicate cancellation of the password entry sequence. In one embodiment, the cancel gesture is a timeout period during which no input gesture is recognized / captured. In another embodiment, , the cancel gesture is one finger touching the gesture input device 16 for a predetermined period of time, e.g., two seconds. In yet another embodiment, the cancel gesture is three fingers touching the gesture input device 16. If the input gesture is a cancel gesture, the software unlock method 300 returns to step 310 to clear the previously entered password symbols 48 and randomly assign the input gestures and their corresponding gesture symbols 52 to the password symbols 50, 56.

[0053] If the input gesture is not a cancel gesture, the software unlock method 300 proceeds to step 318 where the input gesture is converted to an input password symbol 48. Step 320 appends the input password symbol 48 to the end of the sequence entered before the input password symbol 48. A next step 328 compares the sequence of input password symbols 48 entered so far with the stored unlock password. If the stored input password symbols 48 match the stored unlock password in step 330, the software unlock method 300 proceeds to step 332. In step 332, the electronic device 10 is unlocked and the software is operable in a software unlock state 334. If the sequence of input password symbols 48 does not match the stored unlock password in step 330, the software unlock method 300 proceeds to step 336. In step 336, the number of stored input password symbols 48 is compared to the maximum allowed number of input password symbols 48. If the cumulative number of entered password symbols 48 is the maximum allowed number, the software unlock method 300 returns to step 310 and clears the entered password symbols 48. If the cumulative number of entered password symbols 48 is less than the maximum allowed number, the software unlock method 300 returns to step 314 where another input gesture is captured.

[0054] As shown in FIG. 9, in one embodiment, the software unlock method 350 begins with the electronic device 10 in a software locked state 352. In the software locked state 352, the electronic device 10 is in a sleep state and the display is turned off. Step 354 initiates the software unlock method 350 via an input gesture (such as, but not limited to, tapping the input gesture device 16 one or more times) or pressing a button 24 of the electronic device 10. Step 354 may, for example, allocate computer memory to hold captured input gestures or to set initial values ​​to computer variables. Step 355 determines whether the electronic device 10 is in a sleep state. If the electronic device 10 is in a sleep state, the display displays the lock screen 30 in step 356. In step 358, the user 20 initiates a password entry via an input gesture (such as, but not limited to, touching the gesture input device 16 or a virtual touch of the camera 18). Upon initiating password entry, the software unlock method 350 randomly assigns input gestures and corresponding gesture symbols 52 to password symbols 50, 56 in step 360. If, in step 355, the software unlock method 350 determines that the electronic device 10 is not in a sleep state, the software unlock method 350 proceeds to step 360.

[0055] Step 360 also clears any previously entered password symbols 48 and / or input gestures (from a previous attempt to enter a stored unlock password). In a next step 362, the display presents the password entry screen 40. In a next step 364, the input gesture is captured. In step 366, the captured input gesture is compared to a predefined cancel gesture used to indicate cancellation of the input password sequence. In one embodiment, the cancel gesture is a timeout period during which the input gesture is not recognized / captured. In another embodiment, the cancel gesture is one finger touching the gesture input device 16 for a predefined period, e.g., two seconds. In yet another embodiment, the cancel gesture is three fingers touching the gesture input device 16. If the input gesture is a cancel gesture, the software unlock method 350 returns to step 360 to clear any previously entered password symbols 48 and randomly assign the input gesture and its corresponding gesture symbol 52 to the password symbols 50, 56.

[0056] If the input gesture is not a cancel gesture, the software unlock method 350 proceeds to step 368 where the input gesture is converted into an input password symbol 48. Step 370 appends the input password symbol 48 to the end of the sequence of previously entered password symbols 48.

[0057] The next step 372 is to compare the sequence of password symbols 48 entered so far with the stored unlock password. If the accumulated input password symbols 48 match the stored unlock password in step 374, the software unlock method 350 proceeds to step 376. In step 376, the electronic device 10 is unlocked and the software is operable in a software unlocked state 378. If the sequence of input password symbols 48 does not match the stored unlock password in step 374, the software unlock method 350 proceeds to step 380. In step 380, the number of accumulated input password symbols 48 is compared with the maximum allowable number of input password symbols 48. If the accumulated number of input password symbols 48 is the maximum allowable number, the software unlock method 350 returns to step 360 and clears the entered password symbols 48. If the cumulative number of entered password symbols 48 is less than the maximum allowed number, the software unlock method 350 returns to step 364 and another input gesture is captured.

[0058] As shown in FIG. 10, in one embodiment, the software unlocking method 400 starts with the electronic device 10 in a software locked state 402. In the software locked state 402, the electronic device 10 is in a sleep state and the display is turned off. Step 404 starts the software unlocking method 400 via an input gesture (such as one or more taps on the input gesture device 16) or pressing a button 24 on the electronic device 10. Step 404 allocates computer memory or sets initial values ​​for computer variables, etc. Step 405 determines whether the electronic device 10 is in a sleep state. If the electronic device 10 is in a sleep state, the display presents the lock screen 30 in step 406. In step 408, the user 20 initiates a password entry via an input gesture (such as, but not limited to, a touch on the gesture input device 16 or a virtual touch on the camera 18). Upon initiating password entry, in step 410, the software unlocking method 400 randomly assigns the input gesture and the corresponding gesture symbol 52 to a password symbol 50, 56. If in step 405, the software unlocking method 400 determines that the electronic device 10 is not in a sleep state, the software unlocking method 400 proceeds to step 410.

[0059] Step 410 also clears any previously entered password symbols 48 (from a previous attempt to enter a stored unlock password) and / or input gestures. Then, in step 412, the display presents the password entry screen 40. In the next step 414, the input gesture is captured. In step 416, the captured input gesture is compared to a predefined cancel gesture that is used to indicate cancellation of the input password sequence. In one embodiment, the cancel gesture is a timeout period during which no input gesture is recognized / captured. Alternative Implementations In one embodiment, the cancel gesture is one finger touching the gesture input device 16 for a predetermined period of time, e.g., two seconds. In yet another embodiment, the cancel gesture is three fingers touching the gesture input device 16. If the input gesture is a cancel gesture, the software unlock method 400 returns to step 410 to clear the previously entered password symbol 48 and randomly assign the input gesture and its corresponding gesture symbol 52 to the password symbols 50, 56.

[0060] If the input gesture is not a cancel gesture, the software unlock method 400 proceeds to step 418, where the input gesture is converted to an input password symbol 48. Step 420 appends the input password symbol 48 to the end of the sequence of previously input password symbols 48. In step 422, the input password symbol 48 is displayed in the password display area 42 of the password input screen 40. The input password symbol 48 is displayed for a predetermined, usually short, time. In step 424, the display time of the input password symbol 48 is compared to a set / predefined time. If the time that the input password symbol 48 is displayed does not match or exceed the set time, the software unlock method 400 returns to step 422. If the time that the input password symbol 48 is displayed exceeds the set time, the software unlock method 400 executes step 426, where a masking symbol 46 is displayed instead of the input password symbol 48. In one embodiment, step 426 of displaying the masking symbol 46 after the set time occurs simultaneously with step 428, described below. In other words, in one embodiment, the timer of step 426 operates independently of step 428 .

[0061] Next, step 428 compares the sequence of input password symbols 48 entered so far with the stored unlock password. If, in step 430, the accumulated input password symbols 48 match the stored unlock password, the software unlock method 400 proceeds to step 432. In step 432, the electronic device 10 is unlocked and the software is operable in a software unlocked state 434. If, in step 430, the sequence of input password symbols 48 does not match the stored unlock password, the software unlock method 400 proceeds to step 436. In step 436, the number of accumulated input password symbols 48 is compared with the maximum allowable number of input password symbols 48. If the accumulated number of entered password symbols 48 is the maximum allowable number, the software unlock method 400 returns to step 410 and clears the entered password symbols 48. If the cumulative number of entered password symbols 48 is less than the maximum allowed number, the software unlock method 400 returns to step 414 where another input gesture is captured.

[0062] When an input gesture is recognized in various software unlocking methods, the corresponding input password symbol 48 displayed in the password display area 42 of the password input screen 40, 70 may be highlighted in the gesture map area 44 to indicate to the user that it has been entered. The password symbol 48 is recognized by the electronic device 10 as a symbol or character associated with the input gesture. In one embodiment, the highlighting of the input password symbol 48 is the display of a border 54 surrounding the corresponding password symbol 50 and / or password minor symbol 56 in the gesture map area 44. In another embodiment, the highlighting of the input password symbol 48 is a change in color of the password symbol 50 and / or password minor symbol 56 in the gesture map area 44. In yet another embodiment, the highlighting of the input password symbol 48 is the display of the password symbol 50 and / or password minor symbol 56 in the gesture map area 44 with a background color of the gesture that is different from the background color of the remainder of the gesture map area 44. In yet another embodiment, the highlighting of the input password symbol 48 is a flashing display of the password symbol 50 and / or password secondary symbol 56 for a period of time in the gesture map region 44. In another embodiment, the input password symbol 48 is highlighted in the gesture map region 44 in any combination of the above-mentioned methods.

[0063] As shown in FIG. 11, in one embodiment, a password entry screen 500 includes a password display area 502 and a gesture map area 504. In the gesture map area 504, a set of password symbols 520 are displayed together with corresponding gesture symbols 530. An input gesture corresponding to the gesture symbols 530 can be used together with the gesture input device 16 and / or the camera 18 to enter a symbol sequence (e.g., a stored unlock password). The password display area 502 includes a first section 516A for displaying an input password symbol 514. The password display area 502 also includes a second section 516B for selectively displaying a symbol 512 used to indicate that a cancel gesture has been entered. Once an input gesture is recognized and converted into an input password symbol 514, the input password symbol 514 is displayed in the first section 516A of the password display area 502 for a predetermined period of time, e.g., 10 seconds. After the entered password symbol 514 is displayed for a predetermined period of time, the entered password symbol 514 is replaced with a mask symbol 510 which is used to hide the entered password symbol 514 .

[0064] 11, the complete set of password symbols 520 is the set {1, 2, 3, 4, 5, 6, 7, 8, 9, 0}. The password symbols 520 may additionally or alternatively use the set {A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z, +}. Other sets of symbols may also be used. The corresponding input gestures may be indicated by displaying gesture symbols 530 next to the password symbols 520. In one embodiment, the gesture symbols 530 and their corresponding input gestures (input gestures for the gesture input device 16 and / or camera 18) are as follows: "One arrow from left to right" indicates a one finger forward swipe gesture, "one arrow from right to left" indicates a one finger backward swipe gesture, "one filled circle" indicates a one finger tap gesture, "two filled circles" indicates a two finger tap gesture, "two arrows from left to right" indicates a two finger forward swipe, "two arrows from right to left" indicates a two finger backward swipe gesture, "one arrow from bottom to top" indicates a one finger bottom to top swipe gesture, and "one arrow from top to bottom" indicates a one finger top to bottom swipe gesture. "two arrows from bottom to top" indicates a swipe gesture from bottom to top with two fingers, "two arrows from top to bottom" indicates a swipe gesture from top to bottom with two fingers, "a combination of an upper arrow from right to left and a lower arrow from left to right" indicates a swipe gesture of moving one finger from back to front and then from front to back, and "a combination of two arrows arranged in a line facing each other" indicates a swipe gesture of moving two fingers simultaneously from front to back and back to front, respectively (i.e., a horizontal pinch). When using the gesture input device 16, a two-finger input gesture means that two fingers touch the gesture input device 16 at the same time. When using the camera 18, a two-finger input gesture means that two fingers perform an input gesture at the same time. Alternative gesture symbols 530 may be used to correspond to the input gestures.As previously mentioned, other input gestures can be used for other functions during the software unlock method and / or password entry.

[0065] In one embodiment, the background 540 of the password entry screen 500 is transparent or partially transparent and the displayed symbols 510, 514, 512, 520, 530 are opaque or partially opaque. In another embodiment, the background 540 of the password entry screen 500 is opaque or partially opaque and the displayed symbols 510, 514, 512, 520, 530 are transparent or partially transparent.

[0066] 12, in one embodiment, the electronic device 10 may include a general-purpose mobile computing device such as a smartphone. The general-purpose mobile computing device 10 includes a gesture input device 16 configured with a touch screen. The general-purpose mobile computing device 10 also includes a processing unit 14, a camera 18, and a button 24.

[0067] One or more features of the embodiments described herein may be combined to create additional embodiments not shown. Although various embodiments have been described in detail, it should be understood that these are presented by way of example and not limitation. It will be apparent to those skilled in the art that the disclosed subject matter may be embodied in other specific forms, variations, and modifications without departing from the scope, spirit, or essential characteristics thereof. The above-described embodiments should therefore be considered in all respects as illustrative and not restrictive. The scope of the present invention is indicated by the appended claims, and all changes that come within the meaning and range of equivalents thereof are intended to be embraced therein.

Claims

1. A method for unlocking a software function executed by a processing unit in a head mounted display including a processing unit, a gesture input device connected to the processing unit, and a display, the method comprising: detecting an input gesture operable to initiate password input via the gesture input device in a situation where the head mounted display is in a software lock state; randomly assigning a set of predefined input gestures to a set of predefined password symbols, the assigned set of input gestures corresponding to the set of password symbols until the password entry is resumed; displaying a password input screen via the display, the password input screen having a password display area and a gesture map area, the set of assigned input gestures being shown together with the set of password symbols in the gesture map area; Detecting one of the assigned input gestures via the gesture input device; at least temporarily displaying an input password symbol in the password display area in response to the detected input gesture; highlighting the detected input gesture in the gesture map area; via the processing unit, comparing an input password symbol corresponding to the detected input gesture with a stored unlock password; The method for unlocking a software function includes enabling the head mounted display to operate in a software unlocked state if the input password symbol matches the stored unlock password.

2. The method of claim 1 , wherein the gesture input device is at least one of a touchpad, a camera, and a touchscreen.

3. 2. The method of claim 1, wherein the randomly assigned input gesture is selected from the set of a one finger forward swipe gesture, a one finger backward swipe gesture, a one finger tap gesture, a two finger tap gesture, a two finger forward swipe gesture, a two finger backward swipe gesture, a one finger bottom-up swipe gesture, a one finger top-down swipe gesture, a two finger bottom-up swipe gesture, and a two finger top-down swipe gesture.

4. 2. The method of claim 1, further comprising: determining whether the head mounted display is in a sleep state; and if the head mounted display is in the sleep state, displaying a lock screen via the display when the input gesture operable to initiate a password input is detected.

5. 5. The method of claim 4, further comprising: displaying the password entry screen when the input gesture operable to initiate a password entry is detected if the head mounted display is not in the sleep state.

6. 2. The method of claim 1, wherein highlighting the detected input gesture includes displaying a box or circle surrounding the input gesture and an entered password symbol.

7. 2. The method of claim 1, wherein detecting at least two input gestures via the gesture input device includes converting the input gestures into the corresponding input password symbols and adding the input password symbols to a sequence of input password symbols.

8. moreover, providing a number indicating the amount of input password symbols; if the number of input password symbols does not match a stored unlock password, comparing the number of input password symbols to a predefined maximum allowable number of input password symbols; 2. The method of claim 1, further comprising: clearing the input password symbols if the number of input password symbols matches the maximum allowable number of input password symbols; and randomly reassigning the set of input gestures to the set of password symbols.

9. 9. The method of claim 8, further comprising not reassigning the set of predefined input gestures to the predefined password symbols if the entered password symbols do not match the stored unlock password.

10. 2. The method of claim 1, further comprising: upon detecting one of the assigned input gestures via the gesture input device, comparing the detected input gesture with a predefined cancel gesture; and if the detected input gesture matches the cancel gesture, clearing any previously entered password symbols and randomly reassigning the set of input gestures to the set of password symbols.

11. 2. The method of claim 1, further comprising displaying the set of assigned input gestures next to and adjacent to the set of password symbols.

12. A processing unit; a gesture input device connected to the processing unit; a projector coupled to the processing unit and operable to generate an image-bearing light; a planar waveguide assembly operable to transmit the image-bearing light to an eyebox; a display including one or more virtual images transmitted through the planar waveguide assembly; Equipped with the gesture input device is operable to detect an input gesture for initiating password input in a software lock state; the processing unit is operable to randomly assign a set of predefined input gestures to a set of predefined password symbols, the assigned set of input gestures corresponding to the set of password symbols until the password entry is resumed; the display is operable to display a password entry screen, the password entry screen having a password display area and a gesture map area, in which the set of assigned input gestures is shown together with the set of password symbols; the gesture input device is operable to detect one of the assigned input gesters; the display at least temporarily displays an entered password symbol in the password display area in response to the detected input gesture; The display highlights the detected input gesture in the gesture map area.

13. 13. The head mounted display system of claim 12, wherein the display is operable to display the set of predefined input gestures randomly assigned adjacent to the set of predefined password symbols.

14. 2. The method of claim 1, wherein the head mounted display further comprises a projector connected to the processing unit and operable to generate image-bearing light, and a planar waveguide assembly operable to transmit the image-bearing light to an eyebox, the display including one or more virtual images transmitted via the planar waveguide assembly.

15. comparing the input password symbol corresponding to the detected input gesture with the stored unlock password; 2. The method of claim 1, further comprising randomly reassigning the set of input gestures to the set of password symbols if the input password symbols do not match the stored unlock password.

Images