Slippage measurement in a head mounted wearable device

Abstract

Systems and methods for detecting displacement of a frame of a head mounted wearable device are provided. A plurality of capacitive sensors are distributed throughout a frame of the head mounted wearable device. Multi-channel single ended reflection is used to estimate relative directional displacement of the plurality of capacitive sensors. A gaze tracking algorithm and / or variant thereof may be implemented based on incremental capacitive responses detected at one or more of the plurality capacitive sensors indicative of a change in pose of the head mounted wearable device. These incremental changes in capacitive responses may be used to detect a removal of the head mounted wearable device, and also temporary standby position / orientation of the head mounted wearable device, thus eliminating the need for a separate on / off sensor.

Description

SLIPPAGE MEASUREMENT IN A HEAD MOUNTED WEARABLE DEVICEFIELD

[0001] This document relates, generally, to detection of slippage in a head mounted wearable device.BACKGROUND

[0002] Wearable computing devices may include, for example head mounted wearable devices, wrist worn wearable devices, hand worn wearable devices, pendants, and the like. Head mounted wearable computing devices may include, for example, smart glasses, headsets, goggles, ear buds, and the like. Wrist / hand worn wearable computing devices may include, for example, smart watches, smart bracelets, smart rings, and the like. Wearable computing devices may include various types of electronic components that provide for functionality of the wearable computing device. In some situations, movement of the housing or frame of the wearable computing device may affect the functionality of the electronic components and / or the wearable computing device.SUMMARY

[0003] In some aspects, the techniques described herein relate to a computer- implemented method, including: initiating operation of a head mounted wearable device; detecting, by a pose detection system including a plurality of capacitive sensors provided in a frame of the head mounted wearable device and driven by an integrated circuit (IC), a pose of the frame relative to a head of a user of the head mounted wearable device based on data received from the plurality of capacitive sensors via a plurality of channels; detecting a change in the data received via at least one of the plurality of channels; determining an updated pose of the frame relative to the head of the user in response to detecting the change; and adjusting a gaze tracking algorithm implemented by a gaze tracking device of the head mounted wearable device based on the updated pose of the frame.

[0004] In some aspects, the techniques described herein relate to a computer- implemented method, wherein determining the updated pose of the frame relative to the head of the user includes: detecting updated data received from each of the plurality of capacitive sensors via the plurality of channels; comparing the updated data to previous data receivedfrom each of the plurality of capacitive sensors via the plurality of channels; detecting changes in the data received from at least one of the plurality of capacitive sensors based on the comparing; determining an updated pattern defined by the updated data received and the changes; and determining the updated pose of the frame relative to the head of the user based on the updated pattern.

[0005] In some aspects, the techniques described herein relate to a computer- implemented method, wherein: comparing the updated data to previous data includes setting the previous data as a baseline value for each of the plurality of channels, and detecting the changes includes detecting a change from the baseline value for each of the plurality of channels.

[0006] In some aspects, the techniques described herein relate to a computer- implemented method, wherein detecting the change, determining the updated pose, and adjusting the gaze tracking algorithm includes: repeatedly detecting the pose of the frame relative to a head of a user of the head mounted wearable device based on data received from the plurality of capacitive sensors via the plurality of channels; repeatedly detecting changes in the data received via at least one of the plurality of channels; repeatedly determining an updated pose of the frame relative to the head of the user; and repeatedly adjusting the gaze tracking algorithm implemented by the gaze tracking device based on the updated pose of the frame to maintain accuracy of eye tracking performed by the gaze tracking device during operation of the head mounted wearable device.

[0007] In some aspects, the techniques described herein relate to a computer- implemented method, wherein adjusting the gaze tracking algorithm includes: determining whether the updated pose of the frame relative to the head of the user is sustained for greater than or equal to a set period of time; and adjusting a current gaze tracking algorithm implemented by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for greater than or equal to the set period of time.

[0008] In some aspects, the techniques described herein relate to a computer- implemented method, wherein adjusting the gaze tracking algorithm includes: maintaining implementation of the current gaze tracking algorithm by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for less than the set period of time.

[0009] In some aspects, the techniques described herein relate to a computer- implemented method, wherein adjusting the gaze tracking algorithm implemented by the gazetracking device includes: in response to determining that the frame remains within a line of sight of the user in the updated pose of the frame relative to the head of the user, implementing a variant of the gaze tracking algorithm that compensates for a change in position of the gaze tracking device relative to the head of the user in the updated pose of the frame, to maintain accuracy of eye tracking performed by the gaze tracking device in the updated pose of the frame.

[0010] In some aspects, the techniques described herein relate to a computer- implemented method, further including: in response to determining that the frame is outside of the line of sight of the user in the updated pose of the frame relative to the head of the user, and that the data received from at least one of the plurality of capacitive sensors via at least one of the plurality of channels indicates contact with the user, operating the head mounted wearable device in a standby mode of operation and temporarily pausing operation of a current session of the head mounted wearable device.

[0011] In some aspects, the techniques described herein relate to a computer- implemented method, further including: in response to determining that the frame is within the line of sight of the user based on the data received from the plurality of capacitive sensors via the plurality of channels, with the head mounted wearable device in the standby mode: terminating operation of the head mounted wearable device in the standby mode and resuming normal operation of the head mounted wearable device for the user, the user having been previously authenticated in the current session; detecting changes in the data received from the plurality of capacitive sensors via the plurality of channels; determine a new updated pose of the frame relative to the head of the user in response to detecting the changes; and implementing a variant of the gaze tracking algorithm by the gaze tracking device based on the new updated pose of the frame.

[0012] In some aspects, the techniques described herein relate to a computer- implemented method, wherein the IC is a single IC that drives the plurality of capacitive sensors, with the plurality of channels respectively providing for communication of the plurality of capacitive sensors with the single IC.

[0013] In some aspects, the techniques described herein relate to a head mounted wearable device, including: a frame, including: a front frame portion; a first arm portion rotatably coupled to a first end portion of the front frame portion; and a second arm portion rotatably coupled to a second end portion of the front frame portion; a gaze tracking device coupled to the frame and configured to track user eye gaze in a worn state of the head mounted wearable device; a pose detection system including a plurality of capacitive sensorsprovided on the frame, driven by a single integrated circuit (IC) including a plurality of channels respectively corresponding to the plurality of capacitive sensors; and non-transitory, computer-readable medium including a memory storing instructions that, when executed by at least one processor of the head mounted wearable device, cause the at least one processor to: detect changes in data received via at least one of the plurality of channels; determine an updated pose of the frame relative to a head of a user of the head mounted wearable device in response to detecting the changes; and adjust a gaze tracking algorithm implemented by the gaze tracking device based on the updated pose of the frame.

[0014] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein the plurality of capacitive sensors includes at least one of: at least one capacitive sensor provided in the first arm portion; at least one capacitive sensor provided in the second arm portion; at least one capacitive sensor provided in upper orbital portions of rim portions of the front frame portion; at least one capacitive sensor provided in lower orbital portions of the rim portions of the front frame portion.

[0015] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein: at least one of the at least one capacitive sensor provided in the first arm portion or the at least one capacitive sensor provided in the second arm portion maintains contact with the head of the user in a plurality of different poses of the frame relative to the head of the user, and / or at least one of the at least one capacitive sensor provided in the upper orbital portions or the at least one capacitive sensor provided in lower orbital portions of the rim portions of the front frame portion remain spaced apart from the head of the user in the plurality of different poses of the frame relative to the head of the user.

[0016] In some aspects, the techniques described herein relate to a head mounted wearable device to 13, wherein the instructions cause the at least one processor to: detect updated data received from each of the plurality of capacitive sensors via the respective plurality of channels; compare the updated data to previous data received from each of the plurality of capacitive sensors via the respective plurality of channels; detect changes in the data received from at least one of the plurality of capacitive sensors based on the comparing; determine an updated pattern defined by the updated data received and the changes; and determine an updated pose of the frame relative to the head of the user based on the updated pattern.

[0017] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein the instructions cause the at least one processor to: set the previous data as a baseline value for each of the plurality of channels; and detect the changes based onchanges detected from the baseline value for each of the plurality of channels.

[0018] In some aspects, the techniques described herein relate to a head mounted wearable device to 15, wherein the instructions cause the at least one processor to adjust the gaze tracking algorithm implemented by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for greater than or equal to a set period of time.

[0019] In some aspects, the techniques described herein relate to a head mounted wearable device to 16, wherein the instructions cause the at least one processor to maintain implementation of a current gaze tracking algorithm by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for less than a set period of time.

[0020] In some aspects, the techniques described herein relate to a head mounted wearable device any to 17, wherein the instructions cause the at least one processor to: determine that the frame remains within a line of sight of the user in the updated pose of the frame relative to the head of the user; and implement a variant of the gaze tracking algorithm that compensates for a change in position of the gaze tracking device relative to the head of the user in the updated pose of the frame, to maintain accuracy of eye tracking performed by the gaze tracking device in the updated pose of the frame.

[0021] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein the instructions cause the at least one processor to: determine that the frame is outside of the line of sight of the user in the updated pose of the frame relative to the head of the user, and that the data received from at least one of the plurality of capacitive sensors via at least one of the plurality of channels indicates contact with the user; and operate the head mounted wearable device in a standby mode of operation and temporarily pause operation of the head mounted wearable device.

[0022] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein the instructions cause the at least one processor to: determine that the frame is within the line of sight of the user based on the data received from the plurality of capacitive sensors via the plurality of channels, with the head mounted wearable device in the standby mode, terminate operation of the head mounted wearable device in the standby mode and resume normal operation of the head mounted wearable device for the user; detect changes in the data received from the plurality of capacitive sensors via the plurality of channels; determine a new updated pose of the frame relative to the head of the user in response to detecting the changes; and implement a variant of the gaze tracking algorithm bythe gaze tracking device based on the new updated pose of the frame.

[0023] In some aspects, the techniques described herein relate to a head mounted wearable device, wherein the instructions cause the at least one processor to: detect that at least one of the plurality of capacitive sensors has maintained contact with the head of the user since initiation of a current session and authentication of the user for the current session; determine that the head mounted wearable device has not been removed from the head of the user during the current session; and terminate operation of the head mounted wearable device in the standby mode and resume normal operation of the head mounted wearable device in the current session without re-authentication of the user.

[0024] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG. 1 A illustrates an example head mounted wearable device worn by a user.

[0026] FIG. IB is a front view, and FIG. 1C is a rear view, of the example head mounted wearable device shown in FIG. 1 A.

[0027] FIG. ID is a block diagram of the example head mounted wearable device shown in FIGs. 1A-1C.

[0028] FIG. 2A is a perspective view of an example frame of the example head mounted wearable device shown in FIGs. 1 A-1C, including a pose detection system.

[0029] FIG. 2B is a circuit diagram of the pose detection system shown in FIG. 2A.

[0030] FIGs. 3 A-3D illustrate the example frame shown in FIG. 2A, at different positions and orientations on an example head.

[0031] FIG. 4 is a flowchart of an example method.DETAILED DESCRIPTION

[0032] Wearable devices can experience changes in position and / or orientation, or pose, or deformation, while worn by the user. These changes in position and / or orientation, or pose may be from an initial calibration position, or a baseline position detected at the beginning of a session, i.e., when the wearable device is donned by the user and authenticated. Additional / continued changes in position and / or orientation of the wearable device may be experienced while the wearable device is worn. In some situations, functionality of the wearable device can be affected by the changes in position and / ororientation, or pose of the wearable device as worn by the user.

[0033] In a head mounted wearable device, for example, in the form of smart glasses, slippage, or movement, or changes in position and / or orientation, or deformation, can adversely impact functionality such as eye tracking, depending on an amount of displacement and / or a direction of the displacement of the glasses. Movement, or displacement, or slippage of these types of glasses can occur due to, for example, user-initiated adjustment, slippage along the nose bridge due to gravity, movement of the glasses due to regular user movement such as, for example, walking, speaking, and other head movement. In some situations, eye tracking performed by a gaze tracking device of the glasses, in particular, accuracy of the eye tracking, can be adversely impacted, depending on an amount of the movement, or displacement, or slippage, experienced. That is, in some situations, movement of the glasses from an initial, calibrated or baseline pose can present challenges in the performance of feature extraction models implemented to provide for eye / gaze tracking. In an example in which the wearable device, in the form of smart glasses, relies on gaze detection and tracking as a form of user input, this may adversely impact the execution of user commands provided via eye gaze inputs.

[0034] This disclosure relates to systems and methods providing for the detection and measurement of movement, or slippage, of a head mounted wearable device, and in particular, to the detection of movement, or slippage, or changes in position and / or orientation, of a head mounted wearable device, that affects gaze tracking algorithms implemented by the head mounted wearable device. Systems and methods disclosed herein relate to the adjustment of and / or selection of a variant of a gaze tracking algorithm to be implemented by the head mounted wearable device based on the detected movement and / or current pose of the head mounted wearable device relative to a baseline pose of the head mounted wearable device. The adjustment of the gaze tracking algorithm may comprise adjusting at least one parameter of the algorithm. In a system and method, as broadly described herein, movement or slippage of the head mounted wearable device, and in particular, a change in position and / or orientation of the head mounted wearable device, is detected based on data obtained by a plurality of sensors provided on a frame portion of the head mounted wearable device. The plurality of sensors are capacitive sensors distributed throughout a frame of the head mounted wearable device. The movement, or slippage, or deformation of the head mounted wearable device is detected based on a directional displacement of the head mounted wearable device relative to a baseline, or previous pose of the head mounted wearable device based data obtained from the plurality of capacitivesensors distributed throughout the frame of the head mounted wearable device. In some examples, the data provided by the plurality of capacitive sensors provides a multi-channel, single-ended reflection coefficient, that allows for estimated directional relative displacement of the head mounted wearable device.

[0035] In systems and methods, as broadly described herein, an amount, or magnitude of the directional displacement of the head mounted wearable device, for example, relative to a baseline or calibrated position, or a previous position, is used to determine whether a change or adjustment in a variant of a gaze tracking algorithm is to be implemented by the gaze tracking device to compensate for the change and maintain accuracy in the eye tracking performed by the gaze tracking device. Application of a gaze tracking algorithm based on an amount of relative displacement from the baseline, or previous position and orientation, or pose of the head mounted wearable device improves accuracy of eye tracking performed by the gaze tracking device, and improves the detection and processing of user inputs detected based on the gaze tracking. In some examples, the detected relative displacement, or change in position / orientation of the head mounted wearable device and / or the current detected pose of the head mounted wearable device is taken into account in the output of visual content by a display device of the head mounted wearable device, based on a display output algorithm implemented by the head mounted wearable device. In some examples, the detected displacement, and corresponding detected current pose of the head mounted wearable device, is used to detect a worn, or not worn, or standby position of the head mounted wearable device, so that a corresponding operation mode can be implemented by the head mounted wearable device.

[0036] Systems and methods, in accordance with implementations described herein, provide for the detection and measurement of movement, or slippage, or displacement, or deformation of a head mounted wearable device based on data provided by the plurality of capacitive sensors distributed throughout the frame of the head mounted wearable device. In some examples, operation and collection of data by the plurality of capacitive sensors does not rely on illumination. The plurality of capacitive sensors are distributed throughout the frame so as to capture a relatively large range of possible movement, or slippage, or displacement of the head mounted wearable device. The plurality of capacitive sensors can operate substantially continuously without excessive power consumption, thus providing for substantially continuous tracking of the pose of the head mounted wearable device, both to compensate for the change(s) in position / orientation and maintain accuracy in the eye tracking performed by the gaze tracking device and to detect an on or off or standby positionof the head mounted wearable device.

[0037] FIG. 1 A illustrates a user wearing an example head mounted wearable device 100 in the form of smart glasses, or augmented reality glasses, including display capability, eye / gaze tracking capability, and computing and / or processing capability. FIG. IB is a front view, and FIG. 1C is a rear view, of the example head mounted wearable device 100 shown in FIG. 1 A. FIG. ID is a block diagram of the example head mounted wearable device 100. FIG. 2A is a perspective view of the example head mounted wearable device 100, illustrating an example pose detection system 200. FIG. 2B is a circuit diagram of the example pose detection system 200. The features of the example head mounted wearable device 100 are presented for purposes of discussion and illustration. The principles to be described herein can be applied to a head mounted wearable device including more, or fewer features and / or different combinations of features than explicitly shown and / or described herein. Similarly, the principles to be described herein can be applied to other types of wearable device, including more, or fewer features and / or combinations of features.

[0038] The example wearable device 100, in the form of smart glasses as shown in FIGs. 1A-2A, includes a frame 110 including rim portions 103 surrounding glass portions, or lenses 107. In some examples, the lenses 107 are corrective / prescription lenses. In some examples, the lenses 107 are glass portions that do not necessarily incorporate corrective / prescription parameters. In some examples, a bridge portion 109 connects the rim portions 103 of the frame 110. The rim portions 103 and the bridge portion 109 define a front frame portion 102 of the frame 110. An arm portion 130 is coupled, for example, rotatably coupled, to end portions of the front frame portion 102 of the frame 110 at respective hinge portions 140. In some examples, portions of the frame 110, including portions of the rim portions 103 and / or the bridge portion 109 and / or the arm portions 130 define a housing of the example wearable device 100, in which components are received.

[0039] In some examples, the example wearable device 100 includes a display device 104, an audio input device 106 (or microphone), an illumination device 108, a sensing system 111, a control system 112, at least one processor 114, and an outward facing image sensor 116 (for example, a camera). In some examples, the sensing system 111 includes various sensing devices, and the control system 112 includes various control system devices including, for example, the at least one processor 114 operably coupled to the components of the control system 112.

[0040] In some examples, the display device 104 outputs visual content, under the control of the control system 112, for example, at an output coupler 105, so that the visualcontent is visible to the user. In the example shown in FIGs. IB and 1C, the display device 104 is provided in one of the two arm portions 130, simply for purposes of discussion and illustration. Display devices 104 can be provided in each of the two arm portions 130 to provide for binocular output of content. In some examples, the display device 104 is a see- through near eye display. In some examples, the display device 104 is configured to project light from a display source onto a portion of teleprompter glass functioning as a beamsplitter seated at an angle (for example, 30-45 degrees, or other angle based on the overall configuration of the wearable device 100). In some examples, the beamsplitter generates reflection and transmission values that allow light from the display device 104 to be partially reflected while remaining light is transmitted through the beamsplitter. In some examples, this type of optic design allows a user to see both physical items in the ambient environment, for example, through the lenses 107, together with content (for example, digital images, user interface elements, virtual content, and the like) output by the display device 104. In some examples, waveguide optics are used to depict content by the display device 104.

[0041] In some examples, the audio input device 106 detects audio signals, for processing by the at least one processor 114 as an audio input command for execution by the control system 112. In some examples, image data detected by the image sensor 116 is processed by the at least one processor 114 for the detection of gesture inputs to be executed by the control system 112.

[0042] In some examples, movements, or gestures, or changes in head / position and / or orientation detected by position and / or orientation sensors, for example, accelerometers and / or gyroscopes and / or magnetometers and the like of an inertial measurement unit (IMU) 117 are processed as gesture inputs, or head gaze inputs, to be executed by the control system 112. Thus, in some examples, the one or more position and / or orientation sensors at least in part define a head gaze tracking system that detects and tracks head position and head movement. In some examples, touch inputs detected at a touch sensitive surface (not shown in FIGs. IB and 1 C) of the frame 110 are processed by the at least one processor 114 as a touch input command for execution by the control system 112.

[0043] In some examples, the wearable device 100 includes a gaze tracking device 115 to detect and track eye gaze direction and movement. In some examples, data captured by the gaze tracking device 115 is processed by the at least one processor 114 to detect and track gaze direction and movement as a user input for execution by the control system 112. In the example shown in FIGs. IB and 1C, the gaze tracking device 115 is provided in one of the two arm portions 130, and the gaze tracking device 115 is provided in the same armportion 130 as the display device 104, so that user eye gaze can be tracked not only with respect to objects in the physical environment, but also with respect to the content output by the display device 104. In some examples, a gaze tracking device 115 is provided in each of the two arm portions 130 to provide for gaze tracking of each of the two eyes of the user.

[0044] In some examples, the example wearable device 100 includes an audio output device 160 (such as, for example, one or more speakers) provided, for example, in a housing defined by one or both of the arm portions 130 of the frame 110, so that sound output by the audio output device 160 can be directed toward the ear of the user. In some examples, the wearable device 100 includes a power storage device 120, such as, for example, a battery, that provides power to the components of the wearable device 100. In the example shown in FIGs. IB and 1C, the power storage device 120 is included in a housing defined by one of the arm portions 130, simply for purposes of discussion and illustration.

[0045] In some examples, some of the components of the example wearable device 100 described above define an input system that receive, or detect, or sense, inputs to be processed by the control system 112. For example, components of the sensing system and various input sensors of the example wearable device 100 may define an input system of the example wearable device. In some examples, conditions detected by the image sensor 116 and / or gaze / gestures detected by the gaze tracking device 115 and / or the IMU 117 and / or audio inputs detected by the audio input devices 106 and / or touch inputs detected at a touch input surface may be processed as inputs to be executed by the control system 112. In some examples, the display device 104, the audio output device 160, haptic devices (not shown in FIGs. IB and 1 C), and other such devices, form an output system of the example wearable device 100. In some examples, one or more communication modules provide for communication and exchange of information between the example wearable device 100 and other external devices, including the external resources 1100. In some examples, the one or more communication modules provide for communication with other external devices and / or the external resources 1100 via a network 1200. In some examples, one or more memory devices are accessible to the at least one processor 114 and the control system 112 for local storage of information.

[0046] The trend toward a lighter weight, smaller form factor in these types of head mounted wearable devices may make the frame 110 more susceptible to slippage and / or movement and / or displacement and / or deflection and / or deformation as described above. Movement, or slippage, or displacement, or deformation / defl ection can affect the accuracy of eye tracking performed by the gaze tracking device 115. That is, this movement, or slippage,or displacement, or other deformation of the frame 110 can cause a shift in a position and / or orientation of an image sensor of the gaze tracking device 115 relative to the eye(s) of the user (for example, a shift in position, from a previous position of the image sensor relative to the eye(s) of the user), thus impacting the accuracy of eye tracking performed by the gaze tracking device 115. This, coupled with the increasing use of eye / gaze tracking as a user input mode and / or for user interaction with content and / or objects in the physical environment, puts more emphasis on the ability to detect and / or estimate movement and / or slippage and / or displacement and / or deformation of the frame 110. This allows the system to implement the appropriate gaze tracking algorithm for a particular position of the frame 110 relative to the eyes of the user, to compensate for changes in position / orientation of the frame 110 during operation and maintain accuracy in eye tracking even as the frame 110 is displaced. This also allows the system to determine if the movement is intended to place the wearable device 100 in standby state, and to control operations of the wearable device 100 accordingly.

[0047] In some examples, the wearable device 100 includes a pose detection system 200 that detects a position and / or orientation of the frame 110 and / or portions of the frame 110, and in particular, changes in position and / or orientation of the frame 110 and / or portions of the frame 110 from a baseline position and / or orientation and / or from a previous position and / or orientation. In some examples, data obtained by a plurality of sensors 210 of the pose detection system 200 is used to detect changes in the position and / or orientation of the wearable device 100, for example, with respect to a point of reference and / or from a baseline position and / or orientation and / or from a previous position and / or orientation, and the like. In some examples, the plurality of sensors 210 are positioned in and / or on the frame 110, and operate intermittently or substantially continuously, so that data collected by the plurality of sensors 210 can provide for the detection changes in the position and / or orientation (i.e., the pose) of the frame 110 of the wearable device 100.

[0048] In some examples, the data collected by the plurality of sensors 210 is processed by the at least one processor 114, to detect changes in position and / or orientation of the frame 110, and to determine if an adjustment in the implementation of a gaze tracking algorithm employed by the gaze tracking device 115 is warranted. That is, in some situations, a detected displacement, associated with a change in the position and / or orientation of the frame 110 of the wearable device 100, may affect the accuracy of the eye tracking performed by the gaze tracking device 115. In some situations, the detected displacement of the framefield of view of image sensor(s)the gaze tracking device 115, and the gaze tracking device 115 can no longer perform the eye tracking. The pose detection system 200, in accordance with implementations described herein, provides for the collection of data for the detection of displacement, i.e., changes in position and / or orientation, of the frame 110 of the wearable device 100, that triggers implementation of a different or modified variant of the gaze tracking algorithm implemented by the gaze tracking device 115 to regain and / or maintain accuracy in the eye tracking performed by the gaze tracking device 115.

[0049] In the example arrangement shown in FIG. 2A, the pose detection system 200 includes a first sensor 210A provided in a first arm portion 130A of the frame 110, a second sensor 210B, a third sensor 210C, a fourth sensor 210D, a fifth sensor 210E a sixth sensor 21 OF and a seventh sensor 210G in and / or on the front frame portion 102 of the frame 110, and an eighth sensor 21 OH in and / or on the second arm portion of the frame 110. In this example arrangement, the second sensor 210B and the seventh sensor 210G are provided at a position on the front frame portion 102 corresponding to the hinge portions 140 of the frame 110. In this example arrangement, the third sensor 210C and fifth sensor 210E are provided in and / or on upper orbital portions of a first rim portion 103A and a second rim portion 130B, respectively, of the front frame portion 102 of the frame 110. In this example arrangement, the fourth sensor 210D and sixth sensor 21 OF are provided in and / or on a lower orbital portion of the first rim portion 103A and second rim portion 103B, respectively. The number and / or arrangement of the plurality of sensors 210 in the example frame 110 shown in FIG. 2A is provided for purposes of discussion and illustration. The principles to be described herein are applicable to other numbers and / or arrangements of sensors in the frame 110 of the wearable device 100.

[0050] In some examples, data provided by the plurality of sensors 210 of the pose detection system 200 is used to determine a degree, or magnitude, or amount, of displacement, or movement or slippage of the frame 110, for example, relative to previous position and / or orientation, or pose. In some examples, the previous position and / or orientation, or pose, is a baseline position and / or orientation, or a calibration position and / or orientation. In some examples, the previous position and / or orientation, or pose, represents an already displaced position from the baseline, or calibration position of the frame 110. In some examples, the degree, or magnitude, or amount, of displacement, or movement or slippage of the frame 110 determined based on the data provided by the plurality of sensors 210 triggers a selection of a variant of the eye tracking algorithm implemented by the gaze tracking device 115 to regain and / or maintain accuracy in eye tracking performed by the gazetracking device 115. In some examples, data provided by the plurality of sensors 210 is used to determine, or discriminate amongst a worn state, a standby state, and a removed state of the wearable device 100.

[0051] As noted above, in some examples, the plurality of sensors 210 are capacitive sensors that are positioned in and / or on various portions of the frame 110. In some examples, the plurality of sensors 210 form a multichannel arrangement with each of the plurality of sensors 210 being operable over a designated channel, driven by a single integrated circuit (IC) 220. In some examples, the IC 220 is coupled to a portion of the frame 110 defining a ground. In some examples, a position of the IC 220 in the frame 110 corresponds to a portion of the frame 110 that remains in relatively close proximity, or contacts, the head of the user as the frame 110 moves. The distribution of the plurality of sensors 210 throughout the frame 110 provides for multiple sensing points, or detection points, throughout the frame 110 from a single signal generated at the IC 220. The data obtained from the multiple sensing points provided by the plurality of sensors 210 via multiple channels from the IC 220 can be processed to determine a displacement of the frame 110. For example, capacitance levels, and changes in capacitance levels, i.e., increases or decreases in capacitance levels, detected at the plurality of sensors 210 may be indicative of a displacement of the frame 110. In some examples, a pattern collectively formed by the capacitance levels / changes in capacitance levels detected at the plurality of sensors 210 may be indicative of displacement of the frame 110. In some examples, capacitance levels and pattern collectively defined by the capacitance levels / changes in capacitance levels may be processed to define changes in position of the frame 110 along one or more of an X axis, a Y axis, or a Z axis and / or rotation of the frame about one or more of the X axis, the Y axis, or the Z axis.

[0052] As described above, the plurality of sensors 210 are distributed throughout the frame 110, with some of the plurality of sensors 210 positioned in relatively close proximity to the head of the user when the wearable device 100 is worn by the user, and some of the plurality of sensors 210 positioned somewhat spaced apart from the head of the user. In some situations, proximity of the individual sensors of the plurality of sensors 210 to the head of the user depends on a position and orientation, or pose, of the wearable device 100 relative to the face / head of the user.

[0053] FIGs. 3A-3D illustrate the frame 110 positioned on a sample head. FIG. 3 A illustrates the frame 110 at a first position and orientation on the sample head, or at a first pose with respect to the sample head. FIG. 3B illustrates the frame 110 at a second position and orientation, in which the frame 110 has moved in the direction of the arrow Ml relativeto the first position shown in FIG. 3 A. FIG. 3B illustrates the frame 110 at a third position and orientation, in which the frame has moved in the direction of the arrow Ml relative to the first and second positions shown in FIGs. 3A and 3B. FIG. 3B illustrates the frame 110 at a fourth position, in which the wearable device 100 is still worn by the user, but the user has rotated the frame 110 in the direction of the arrow M3, temporarily moving the wearable device 100 out of the line of sight of the user.

[0054] For purposes of discussion and illustration, the arrangement shown in FIG. 3A will be considered an initial position, or a baseline position, or a calibration position, representing an initial as-worn position and orientation of the frame 110 on the head of the user. For example, when donning the wearable device 100 and initiating operation, the initial, as worn, or baseline or calibration position and / or orientation of the frame 110 may be detected. For example, the initial, as worn, or baseline, or calibration position and / or orientation may be detected as the wearable device 100 is authenticated for use by the user during a particular session. In some examples, a single excitation, or signal, generated at the IC 220 is emitted at each of the plurality of sensors 210. An electrical field emitted from the plurality of sensors 210 (in the form of capacitive sensors) can interact, and detect the proximity of a capacitive object, in this example, in the form of the body or skin of the user.

[0055] In this example, the disruption of the electrical field due to the presence of the body, or the skin, of the user, and received or returned to the plurality of sensors 210 (in the form of capacitive sensors) in the form of a corresponding capacitive response can be captured at the first position shown in FIG. 3 A. The capacitive responses detected at each of the plurality of sensors 210 are associated with the first position, establishing a baseline pattern of capacitive responses at the plurality of sensors 210 associated with the first position. In some examples, this initial, or baseline position is detected as part of a user authentication and / or pairing process when initiating an operating session of the wearable device 100. The first position can be considered, for example, an initial, as-worn position and orientation, or pose, of the wearable device 100 for the particular user for which the authentication has been carried out for that particular session.

[0056] In some examples, capacitance levels at the plurality of sensors 210 are monitored, to detect changes, or fluctuations, in the individual capacitance levels. Changes in the capacitance levels, for example, from a previous capacitance level for the respective sensor 210 (including an initial, or baseline capacitance level) may be indicative of a movement, or slippage, or displacement of the frame 110. In some examples, a change in a pattern of the collective capacitance levels of the plurality of sensors 210 may be indicativeof a movement, or slippage, or displacement of the frame 110. Continued monitoring of the capacitance levels may provide an indication as to whether the detected changes in capacitance levels represent a temporal movement, or slippage, or displacement of the frame 110, or a sustained change in state of the frame 110 and / or operation of the wearable device 100.

[0057] In some examples, the IC 220 is fixed to a chassis that provides a ground for electronic components received in an installation space within the frame 110. When worn by the user, the chassis of the wearable device 100 may be in contact with the head of the user, generating a relatively large capacitance. The plurality of sensors 210 are connected to the IC 220, but not to the chassis. The (e.g., incremental) capacitance levels experienced at the plurality of sensors 210, arranged in series with the relatively large capacitance at the contact between the chassis and the head of the user, can be processed, for example, by an analog to digital controller of the control system. These capacitance values can be intermittently, or substantially continuously, processed to detect changes in capacitance values at the plurality of sensors 210.

[0058] In some examples, the capacitive responses detected at the plurality of sensors 210, and changes in capacitive responses at the plurality of sensors 210, can be correlated into a pattern, and change in pattern, for example, relative to a set reference point. For example, the pattern may include data representing the position of the sensors 210 or at least of some of the sensors 210, in particular relative to the set reference point (e.g., reference point R in Fig. 3 A). The change pattern may provide an indication of a change in position and / or orientation, or pose, of the frame 110 on the head / face of the user. In some examples, the set reference point corresponds to the ground connection of the IC 220 to the chassis of the frame 110 of the wearable device 100. In some examples, the reference point may correspond to another set point.

[0059] In FIG. 3 A, a pattern Pl is defined based on the capacitive responses detected at the plurality of sensors 210, for example, compared to the capacitive response at the IC 220 coupled to the ground on the chassis. The capacitive levels at the plurality of sensors 210 relative to the capacitive level detected at the ground may be processed to be correlated to a respective plurality of distances represented by vectors R1 from a set reference point R. In some examples, the set reference point is at the ground provided by the chassis, in contact with the head of the user, and that remains in contact with the head of the user, even in response to movement, or displacement of the frame 110 on the head of the user. In some examples, the set reference point is a point at a set distance and X / Y / Z position relative to theground provided by the chassis, that remains in contact with the head of the user. In the example arrangement shown in FIG. 3 A, an example set reference point R is provided to facilitate discussion and illustration.

[0060] In some examples, the capacitance levels detected at the plurality of sensors 210 are processed to, collectively, define the first pattern Pl that is representative of a pose, i.e., a position and orientation, of the frame 110 in the first position on the face of the user. That is, each incremental capacitance level (i.e., incremental from the capacitance level at the ground in contact with the head of the user), taken with the known distribution of the plurality of sensors 210 throughout the frame 110, is correlated with a distance, i.e., an X / Y / Z distance from the set reference point R. In this example arrangement, some of the plurality of sensors 210 are at different vertical positions in the front frame portion 102 of the frame 110, some of the plurality of sensors 210 are at different horizontal positions along the front frame portion 102 of the frame 110, and some of the plurality of sensors 210 are provided in each of the arm portions 130 of the frame 110. This allows the system to establish an initial, or baseline position and orientation, or pose of the frame 110 (and each of the plurality of sensors 210) relative to the set reference point R. In some examples, vectors R1 represent positions (i.e., X / Y / Z positions) of the plurality of sensors 210 relative to the set reference point R, based on a magnitude of the incremental capacitance levels detected at the plurality of sensors 210.

[0061] In response to detection of the head, and of the pose of the frame 110 on the face of the user (in this particular example, the initial position / orientation, or baseline position / orientation, or calibration position / orientation), and detection of initiation of use by a user, an eye tracking algorithm is implemented by the gaze tracking device 115. In some examples, a version, or variant, of the eye tracking algorithm to be implemented by the gaze tracking device 115 is selected based on the detected pose, i.e., position / orientation of the frame 110. As the wearable device 100 is operable, the system monitors the plurality of channels respectively associated with the plurality of sensors 210, and the data collected at the plurality of sensors 210, to detect changes in the data collected by the plurality of sensors 210. In this example, the data obtained by the plurality of sensors 210 is processed to detect changes in the data collected through the plurality of data channels associated with the plurality of sensors 210 that are indicative of a change in state of the wearable device 100. A change in state may include, for example, a change in position / orientation, or pose, of the wearable device 100 relative to the head of the user that may impact the accuracy of eye tracking performed by the gaze tracking device 115 implementing the current variant of thegaze tracking algorithm. In some examples, the change in state may include a change in position / orientation, or pose, of the wearable device 100 relative to the head of the user that indicates the user has moved the wearable device 100 to a standby position, in which the control system of the wearable device 100 may temporarily pause execution of some of the processes executing on the wearable device 100 during the current session. In some examples, the change in state may include a change in position / orientation, or pose, of the wearable device 100 relative to the head of the user that indicates the user has removed the wearable device 100, causing the wearable device 100 to terminate the current session.

[0062] In some examples, changes in position and / or orientation, or pose, of the frame 110 of the wearable device 100 include a movement of the frame 110 (for example, from the initial, or baseline position shown in FIG. 3D, or another previously established position / orientation of the frame 110) along one of the X axis, the Y axis and / or the Z axis, and / or combinations thereof. In some examples, changes in position and / or orientation, or pose, of the frame 110 of the wearable device 100 may represent a rotational movement of the frame 110 in the direction of the arrow Al about the X axis and / or a rotational movement in the direction of the arrow A2 about the Y axis and / or a rotational movement in the direction of the arrow A3 about the Z axis.

[0063] FIG. 3B illustrates the frame 110 at a second position and orientation on the sample head, or at a second pose with respect to the sample head. In the second position and orientation, the frame 110 has moved in the direction of the arrow Ml from the first, or baseline, or as-worn position and orientation shown in FIG. 3 A. The example movement, from the first position shown in FIG. 3 A to the second position shown in FIG. 3B, may represent, for example, movement along the Y axis and / or the Z axis, simply for purposes of discussion and illustration, as well as some rotational movement about one of the X and / or Y and / or Z axes. This movement may be in response to, for example, an adjustment by the user of the position and / or orientation of the frame 110 to a more comfortable position, a movement of the frame 110 in response to user head movement, and the like. This movement of the frame 110 may be detected based on the data collected and analyzed as described above. The movement (i.e., from the first position / orientation to the second position / orientation in this example) may be detected in response to a detected change in the pattern of the data returned through the multiple channels respectively corresponding to the plurality of sensors 210.

[0064] In FIG. 3B, a pattern P2 is defined based on incremental changes detected in the capacitive responses at the plurality of sensors 210, for example, compared to thecapacitive response at the previous, first position shown in FIG. 3 A. As described above, current capacitance levels detected at the plurality of sensors 210 may be compared to previous capacitance levels detected at the plurality of sensors 210 in the first position, or previous position, shown in FIG. 3 A, to detect incremental changes in one or more of the capacitance levels. For example, a capacitive response, or capacitive level, at each of the plurality of sensors 210 in the first position shown in FIG. 3 A may be considered a baseline level, or a zero-set level, with further monitoring detecting incremental increases and / or decreases from those baseline, or zero-set levels (i.e., rather than measuring absolute capacitance levels). In FIG. 3B, the second pattern P2 is shown in dotted lines. The first pattern Pl is shown in solid lines, to provide a basis of comparison, and illustrate the movement of the frame 110 and corresponding change in the pattern based on the detected incremental changes in capacitance at the plurality of sensors 210. The incremental changes in capacitive levels at the plurality of sensors 210 relative to the capacitive levels detected at the previous, first position, may be correlated to a respective plurality of distances represented by vectors R2 from the set reference point R, that define corresponding points of the second pattern P2.

[0065] In some examples, the capacitance levels detected at the plurality of sensors 210 are processed to, collectively, define the second pattern P2 that is representative of a pose, i.e., a position and orientation, of the frame 110 in the second position on the face of the user. That is, each incremental capacitance level (i.e., incremental from the baseline capacitance level at the first position), taken with the known distribution of the plurality of sensors 210 throughout the frame 110, is correlated with a distance, i.e., an X / Y / Z distance from the set reference point R. In some examples, vectors R2 represent positions (i.e., X / Y / Z positions) of the plurality of sensors 210 relative to the set reference point R, based on a magnitude of the incremental changes in capacitance levels detected at the plurality of sensors 210.

[0066] In some examples, a determination is made as to whether the detected incremental changes in capacitance and corresponding detected change in pose indicates a change state in of the wearable device 100, including a change in state that may warrant selection of a new variant of the gaze tracking algorithm, or a change in state that indicates the user has moved the wearable device 100 to a standby position, or a change in indicating the user has removed the wearable device 100.

[0067] In some examples, the detected change in pattern, i.e., from the first pattern Pl to the second pattern P2, associated with the change in position / orientation, may becharacterized as a change in state of the wearable device 100 that may affect accuracy of eye tracking performed by the gaze tracking device 115. The determination that the change may affect accuracy of the eye tracking performed by the gaze tracking device 115 may be made based on, for example, a degree or magnitude of change detected from the baseline, first position. In this situation, the system may use the detected change to select a variant of the eye tracking algorithm to be implemented by the gaze tracking device 115 to accommodate the change in position and / or orientation of the frame 110 of the wearable device 100.

[0068] In some examples, a temporal factor associated with the detected change may be taken into account in determining that the detected change may affect accuracy of the eye tracking performed by the gaze tracking device 115. In some situations, the change in position and / or orientation of the frame 110 of the wearable device 100 may be temporary, and not a sustained change in position and / or orientation. For example, user actions such as speaking, chewing, walking and other inadvertent user movement may cause temporary shifts in position and / or orientation of the frame 110. The temporal nature of these changes may be reflected in the data collected via the plurality of data channels, with the position, or pose, of the frame 110 returning to at or within a set range or threshold of the previous pose, or first pose, within a set period of time. Thus, in some examples, a variant of the gaze tracking algorithm for implementation by the gaze tracking device 115 is selected after the incremental changes in the capacitance levels detected by the plurality of sensors 210 indicating the detected change in position and / or orientation of the wearable device 100 are sustained for greater than or equal to a set threshold period of time. Similarly, in some examples, one or more display output algorithms may be adjusted based on the magnitude of the change in position and / or orientation, or pose, of the frame 110, and a determination that the change is sustained , and not temporary. This may avoid unnecessary and / or ineffective changes in the implementation of gaze tracking and / or the output of display content that could otherwise be disruptive to the user interaction with the wearable device 100 when the change in position of the frame 110 is temporary

[0069] FIG. 3C illustrates the frame 110 at a third position and orientation on the sample head, or at a third pose with respect to the sample head. In the third position and orientation, the frame 110 has moved in the direction of the arrow M2 relative to the first, or baseline, or as-worn position and orientation shown in FIG. 3 A, and also relative to the second position and orientation shown in FIG. 3B. This movement may represent, for example, movement along the Y axis and / or the Z axis, simply for purposes of discussion and illustration, as well as movement along the X axis and / or some rotational movement aboutone of the X and / or Y and / or Z axes. This movement may represent, for example, slippage of the frame 110 down along the nose bridge of the user due to various different factors including, for example, a fit of the frame 110 on the user’s head / facial features, movement of the user’s head, or other user activity. This movement of the frame 110 may be detected based on the data collected and analyzed as described above. The movement (i.e., from the first and / or second position / orientation to the third position / orientation in this example) may be detected based on an incremental change in the capacitance levels at the plurality of sensors 210 relative to the capacitance levels at the previous, or baseline capacitance levels.

[0070] In FIG. 3C, a pattern P3 is defined based on incremental changes detected in the capacitive responses at the plurality of sensors 210, for example, compared to the capacitive response at the previous position (for example, the first position shown in FIG. 3 A or the second position shown in FIG. 3B). As described above, current capacitance levels detected at the plurality of sensors 210 may be compared to previous capacitance levels detected at the plurality of sensors 210 at the previous position, to detect incremental changes in one or more of the capacitance levels. In FIG. 3C, the third pattern P3 is shown in dashed lines. The first pattern Pl is shown in solid lines, and the second pattern is shown in dotted lines, to provide a basis of comparison, and illustrate the movement of the frame 110 and corresponding change in the pattern based on the detected incremental changes in capacitance at the plurality of sensors 210. The incremental changes in capacitive levels at the plurality of sensors 210 relative to the capacitive levels detected at the previous position, may be correlated to a respective plurality of distances represented by vectors R3 from the set reference point R that define corresponding points of the third pattern P3.

[0071] In some examples, the capacitance levels detected at the plurality of sensors 210 are processed to, collectively, define the third pattern P3 that is representative of a pose, i.e., a position and orientation, of the frame 110 in the third position on the face of the user. That is, each incremental change in capacitance level from the respective capacitance level at the previous position, taken with the known distribution of the plurality of sensors 210 throughout the frame 110, is correlated with a distance, i.e., an X / Y / Z distance from the set reference point R. In some examples, the vectors R3 represent positions (i.e., X / Y / Z positions) of the plurality of sensors 210 relative to the set reference point R, based on a magnitude of the incremental changes in capacitance levels detected at the plurality of sensors 210.

[0072] In some examples, a determination is made as to whether the detected incremental changes in capacitance and corresponding detected movement of the frame 110to the third position indicates a change in state of the wearable device 100, including a change in state that may warrant selection of a new variant of the gaze tracking algorithm, or a change in state that indicates the user has moved the wearable device 100 to a standby position, or a change in indicating the user has removed the wearable device 100.

[0073] In some examples, the detected change, i.e., the third pattern associated with the third position, and change in position / orientation of the frame 110, may be characterized as a change in state of the wearable device 100 that may affect accuracy of eye tracking performed by the gaze tracking device 115. The determination that the change may affect accuracy of the eye tracking performed by the gaze tracking device 115 may be made based on, for example, a degree or magnitude of change detected from previous position. In this situation, the system may use the detected change to select a variant of the eye tracking algorithm to be implemented by the gaze tracking device 115 to accommodate the change in position and / or orientation of the frame 110 of the wearable device 100.

[0074] Again, a temporal factor associated with the detected change may be taken into account in determining that the detected change may affect accuracy of the eye tracking performed by the gaze tracking device 115. As noted above, in some examples, selection of a different variant of the gaze tracking algorithm for implementation by the gaze tracking device 115 is selected in response to a determination that the change in state is sustained for greater than or equal to a set threshold period of time. Similarly, in some examples, one or more display output algorithms may be adjusted based on the magnitude of the change in position and / or orientation, or pose, of the frame 110, and a determination that the change is sustained , and not temporary. This may avoid unnecessary and / or ineffective changes in the implementation of gaze tracking and / or the output of display content that could otherwise be disruptive to the user interaction with the wearable device 100 when the change in position of the frame 110 is temporary.

[0075] FIG. 3D illustrates the frame 110 at a fourth position and orientation on the sample head, or at a fourth pose with respect to the sample head. In the third position and orientation, the frame 110 has been rotated in the direction of the arrow M3 relative to the first and / or second and / or third positions. This movement may represent, for example, movement along the Y axis and / or the Y axis and / or the Z axis, as well as rotational movement in the direction of the arrow Al about the X axis. This movement of the frame 110 may be detected based on the data collected and analyzed as described above. The movement to the fourth position may be detected based on an incremental change in the capacitance levels at the plurality of sensors 210 relative to the capacitance levels at the previous, orbaseline capacitance levels.

[0076] In some examples, the user may move the wearable device 100 to the fourth position to place the wearable device 100 in a standby mode without removing the wearable device 100 from the head of the user. In some examples, movement of the wearable device 100 to the fourth position and into the standby mode can cause some operations to be temporarily paused, or to pause a current session, while in the standby mode, thus preserving battery power. In some examples, the user can resume normal operation of the head mounted wearable device 100, resuming the paused experiences, or resuming the previously paused session, when the pose detection system 200 detects that the wearable device 100 has been moved back into the line of sight of the user. In some examples, the user may resume normal operation of the head mounted wearable device, resuming the paused session without having to re-authenticate and / or repeat pairing. That is, as at least some of the plurality of sensors 210 have maintained contact with the head of the user, the pose detection system 200 can positively determine that, while the wearable device 100 was moved (and transitioned to standby mode), the wearable device 100 did not leave the head of the originally authenticated user.

[0077] In FIG. 3D, a pattern P4 is defined based on incremental changes detected in the capacitive responses at the plurality of sensors 210, for example, compared to the capacitive response at the previous position. As described above, current capacitance levels detected at the plurality of sensors 210 may be compared to previous capacitance levels detected at the plurality of sensors 210 at the previous position, to detect incremental changes in one or more of the capacitance levels. In FIG. 3D, the fourth pattern P4 is shown in alternating dash and dotted line. The first pattern Pl is shown in solid lines, the second pattern is shown in dotted lines, and the third pattern is shown in dashed lines, to provide a basis of comparison, and illustrate the movement of the frame 110 and corresponding change in the pattern based on the detected incremental changes in capacitance at the plurality of sensors 210. The incremental changes in capacitive levels at the plurality of sensors 210 relative to the capacitive levels detected at the previous position, may be correlated to a respective plurality of distances represented by vectors R4 from the set reference point R that define corresponding points of the fourth pattern P4.

[0078] In some examples, the capacitance levels detected at the plurality of sensors 210 are processed to, collectively, define the fourth pattern P4 that is representative of a pose, i.e., a position and orientation, of the frame 110 in the fourth position on the head of the user. That is, each incremental change in capacitance level from the respective capacitance level atthe previous position, taken with the known distribution of the plurality of sensors 210 throughout the frame 110, is correlated with a distance, i.e., an X / Y / Z distance from the set reference point R. In some examples, the vectors R4 represent positions (i.e., X / Y / Z positions) of the plurality of sensors 210 relative to the set reference point R, based on a magnitude of the incremental changes in capacitance levels detected at the plurality of sensors 210.

[0079] In the fourth position, the changes in capacitance at some of the plurality of sensors 210 correspond to a large change in position relative to the reference point R. For example, in the fourth position, the second sensor 210B, third sensor 210C, fourth sensor 210D, fifth sensor 210E sixth sensor 21 OF and seventh sensor 210G on the front frame portion 102 of the frame 110 have experienced significant movement and are considerably closer to the reference point R than in any of the previous first and / or second and / or third positions in which the front frame portion 102 is in the line of sight of the user. However, the first sensor 210A and the eighth sensor 21 OH have maintained a somewhat consistent position relative to the reference point R, and have maintained contact with the head of the user, providing positive confirmation that the wearable device 100 has moved but remains on the head of the user, and has not been removed. Thus, the system may put the wearable device 100 in a standby mode of operation, pausing a current user session and / or execution of current tasks and / or experiences. This may allow the user to resume normal operation of the head mounted wearable device, resuming activity in the session at a point at which the experiences were paused or interrupted, providing for continuity in the user experience. As the system can positively confirm that the wearable device 100 has not been removed from the head of the authenticated user that initiated the session, authentication can be maintained, and the user does not have to re-authenticate and / or re-pair with other external devices when moving the frame 110 back into their line of sight and resuming the session. Thus, in some examples, the pose detection system 200 also incorporates the features of an on / off detector that detects whether the wearable device 100 is in a worn or removed state, thus eliminating the need for a separate on / off sensor that is substantially continuously operating and consuming power and space.

[0080] FIG. 4 is a flowchart of an example method 400, in accordance with implementations described herein. The example method may be performed by computing systems implemented in the example head mounted wearable device 100 described above, or other such device having processing / computing capability and / or display capability and / or eye / gaze tracking capability.

[0081] A user dons a wearable device, such as the example head mounted wearable device 100, or other wearable device, and initiates operation (block 410). The user initiates a session (block 420). In a head mounted wearable device, this can include initiating gaze tracking capability. The wearable device performs an authentication process (block 420) to, for example, establish a user identity, associate the session with a particular user profile, preferences, and the like. In some examples, authentication is performed using the gaze tracking capability to perform an iris scan. In some examples, authentication includes additional data collection and / or additional user inputs. In some examples, pairing with an external device is also performed (block 430). In a head mounted wearable device with a pose detection system including a plurality of sensors, such as the example pose detection system 200 described above, a n initial position and / or orientation, or pose of the wearable device is detected based on a pattern of the data returned from the plurality of sensors arranged as described above (block 440). The pattern is indicative of a position / orientation of the wearable device as worn by the user.

[0082] The system may continue to monitor the plurality of channels associated with the plurality of sensors of the pose detection system to detect changes in the data returned via the plurality of channels (block 450). As the plurality of channels are monitored, it is determined whether or not one or more of the plurality of channels (corresponding to the plurality of sensors) returns a data value representing a change (e.g., an incremental change) from a previous data value for the respective data channel / sensor (block 460). In some examples, this determination is made using the previous data (associated with the current pattern). In response to a determination that at least one of the plurality of channels returns data that represents an incremental change, a new, or updated pattern is determined (block 470), the pattern, e.g., representing current positions of at least some of the sensors. The new, or updated pattern is representative of a new, or updated, position and / or orientation, or pose, of the wearable device compared to the previous, position and / or orientation, or pose.

[0083] In response to a determination that the new, or updated pattern represents a position / orientation of the wearable device in which the wearable device has been moved out of the line of sight of the user (block 480), the wearable device may implement standby mode of operation (block 485). For example, as described above with respect to FIG. 3D, the standby mode of operation may be implemented when the new / updated pattern indicates that a portion of the wearable device has been moved out of the line of sight of the user, while a portion of the wearable device maintains contact with the head of the user, confirming that the wearable device has not been removed from the head of the user. In the standby mode ofoperation, tasks and / or experiences executing on the wearable device during the current session may be temporarily paused or suspended, to preserve power, allowing the user to resume normal operation, and resume interaction at the point at which the tasks and / or experiences were paused when the session is resumed and normal operation of the head mounted wearable device 100 is resumed.

[0084] In response to a determination that the that the new, or updated pattern represents a position / orientation of the wearable device in which the wearable device remains in the line of sight of the user (block 480), it is then determined whether or not the new, or updated pattern is sustained for a period of time that is greater than or equal to a threshold period of time (block 490). That is, it may be determined that the new / updated pattern represents a temporary change in position / orientation of the wearable device when the data obtained via the plurality of channels indicates the new / updated pattern is not sustained for greater than or equal to the threshold period of time. It may be determined that the new / updated pattern represents a sustained change in position / orientation of the wearable device when the data obtained via the plurality of channels indicates the new / updated pattern is sustained for greater than or equal to the threshold period of time. In response to determining that the new / updated pattern represents a sustained change in the position / orientation of the wearable device, a variant of a gaze tracking algorithm may be implemented by a gaze tracking device of the wearable device that will provide for accurate gaze tracking at the new position / orientation, or pose of the wearable device (block 495). The process may continue until the session is terminated.

[0085] Further to the descriptions above, a user may be provided with controls allowing the user to make an election as to both if and when systems, programs, or features described herein may enable collection of user information (e.g., information about a user’s social network, social actions, or activities, profession, a user’s preferences, or a user’s current location), and whether or not the user is sent content or communications from a server. In addition, certain data may be treated in one or more ways before it is stored or used, so that personally identifiable information is removed. For example, a user’s identity may be treated so that no personally identifiable information can be determined for the user, or a user’s geographic location may be generalized where location information is obtained (such as to a city, zip code, or state level), so that a particular location of a user cannot be determined. Thus, the user may have control over what information is collected about the user, how that information is used, and what information is provided to the user.

[0086] The terminology used herein is for the purpose of describing particularimplementations only and is not intended to be limiting of the implementations. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and / or "including," when used in this specification, specify the presence of the stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0087] It will be understood that when an element is referred to as being "coupled," "connected," or "responsive" to, or "on," another element, it can be directly coupled, connected, or responsive to, or on, the other element, or intervening elements may also be present. In contrast, when an element is referred to as being "directly coupled," "directly connected," or "directly responsive" to, or "directly on," another element, there are no intervening elements present. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0088] Spatially relative terms, such as "beneath," "below," "lower," "above," "upper," and the like, may be used herein for ease of description to describe one element or feature in relationship to another element(s) or feature(s) as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if the device in the figures is turned over, elements described as "below" or "beneath" other elements or features would then be oriented "above" the other elements or features. Thus, the term "below" can encompass both an orientation of above and below. The device may be otherwise oriented (rotated 70 degrees or at other orientations) and the spatially relative descriptors used herein may be interpreted accordingly.

[0089] Example implementations of the concepts are described herein with reference to cross-sectional illustrations that are schematic illustrations of idealized implementations (and intermediate structures) of example implementations. As such, variations from the shapes of the illustrations as a result, for example, of manufacturing techniques and / or tolerances, are to be expected. Thus, example implementations of the described concepts should not be construed as limited to the particular shapes of regions illustrated herein but are to include deviations in shapes that result, for example, from manufacturing. Accordingly, the regions illustrated in the figures are schematic in nature and their shapes are not intended to illustrate the actual shape of a region of a device and are not intended to limit the scope of example implementations.

[0090] It will be understood that although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. Thus, a "first" element could be termed a "second" element without departing from the teachings of the present implementations.

[0091] Unless otherwise defined, the terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which these concepts belong. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and / or the present specification and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0092] While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover such modifications and changes as fall within the scope of the implementations. It should be understood that they have been presented by way of example only, not limitation, and various changes in form and details may be made. Any portion of the apparatus and / or methods described herein may be combined in any combination, except mutually exclusive combinations. The implementations described herein can include various combinations and / or sub-combinations of the functions, components, and / or features of the different implementations described.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method, comprising: initiating operation of a head mounted wearable device; detecting, by a pose detection system including a plurality of capacitive sensors provided in a frame of the head mounted wearable device and driven by an integrated circuit (IC), a pose of the frame relative to a head of a user of the head mounted wearable device based on data received from the plurality of capacitive sensors via a plurality of channels; detecting a change in the data received via at least one of the plurality of channels; determining an updated pose of the frame relative to the head of the user in response to detecting the change; and adjusting a gaze tracking algorithm implemented by a gaze tracking device of the head mounted wearable device based on the updated pose of the frame.

2. The computer-implemented method of claim 1, wherein determining the updated pose of the frame relative to the head of the user includes: detecting updated data received from each of the plurality of capacitive sensors via the plurality of channels; comparing the updated data to previous data received from each of the plurality of capacitive sensors via the plurality of channels; detecting changes in the data received from at least one of the plurality of capacitive sensors based on the comparing; determining an updated pattern defined by the updated data received and the changes; and determining the updated pose of the frame relative to the head of the user based on the updated pattern.

3. The computer-implemented method of claim 2, wherein: comparing the updated data to previous data includes setting the previous data as a baseline value for each of the plurality of channels, and detecting the changes includes detecting a change from the baseline value for each of the plurality of channels.

4. The computer-implemented method of any of the preceding claims, wherein detecting the change, determining the updated pose, and adjusting the gaze tracking algorithm comprises: repeatedly detecting the pose of the frame relative to a head of a user of the head mounted wearable device based on data received from the plurality of capacitive sensors via the plurality of channels; repeatedly detecting changes in the data received via at least one of the plurality of channels; repeatedly determining an updated pose of the frame relative to the head of the user; and repeatedly adjusting the gaze tracking algorithm implemented by the gaze tracking device based on the updated pose of the frame to maintain accuracy of eye tracking performed by the gaze tracking device during operation of the head mounted wearable device.

5. The computer-implemented method of any of the preceding claims, wherein adjusting the gaze tracking algorithm includes: determining whether the updated pose of the frame relative to the head of the user is sustained for greater than or equal to a set period of time; and adjusting a current gaze tracking algorithm implemented by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for greater than or equal to the set period of time.

6. The computer-implemented method of claim 5, wherein adjusting the gaze tracking algorithm includes: maintaining implementation of the current gaze tracking algorithm by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for less than the set period of time.

7. The computer-implemented method of any of the preceding claims, wherein adjusting the gaze tracking algorithm implemented by the gaze tracking device includes: in response to determining that the frame remains within a line of sight of the user in the updated pose of the frame relative to the head of the user, implementing a variant of the gaze tracking algorithm that compensates for a change in position of the gaze tracking devicerelative to the head of the user in the updated pose of the frame, to maintain accuracy of eye tracking performed by the gaze tracking device in the updated pose of the frame.

8. The computer-implemented method of claim 7, further comprising: in response to determining that the frame is outside of the line of sight of the user in the updated pose of the frame relative to the head of the user, and that the data received from at least one of the plurality of capacitive sensors via at least one of the plurality of channels indicates contact with the user, operating the head mounted wearable device in a standby mode of operation and temporarily pausing operation of a current session of the head mounted wearable device.

9. The computer-implemented method of claim 8, further comprising: in response to determining that the frame is within the line of sight of the user based on the data received from the plurality of capacitive sensors via the plurality of channels, with the head mounted wearable device in the standby mode: terminating operation of the head mounted wearable device in the standby mode and resuming normal operation of the head mounted wearable device for the user, the user having been previously authenticated in the current session; detecting changes in the data received from the plurality of capacitive sensors via the plurality of channels; determine a new updated pose of the frame relative to the head of the user in response to detecting the changes; and implementing a variant of the gaze tracking algorithm by the gaze tracking device based on the new updated pose of the frame.

10. The computer-implemented method of claim of any of the preceding claims, wherein the IC is a single IC that drives the plurality of capacitive sensors, with the plurality of channels respectively providing for communication of the plurality of capacitive sensors with the single IC.

11. A head mounted wearable device, comprising: a frame, including: a front frame portion;a first arm portion rotatably coupled to a first end portion of the front frame portion; and a second arm portion rotatably coupled to a second end portion of the front frame portion; a gaze tracking device coupled to the frame and configured to track user eye gaze in a worn state of the head mounted wearable device; a pose detection system including a plurality of capacitive sensors provided on the frame, driven by a single integrated circuit (IC) including a plurality of channels respectively corresponding to the plurality of capacitive sensors; and non-transitory, computer-readable medium including a memory storing instructions that, when executed by at least one processor of the head mounted wearable device, cause the at least one processor to: detect changes in data received via at least one of the plurality of channels; determine an updated pose of the frame relative to a head of a user of the head mounted wearable device in response to detecting the changes; and adjust a gaze tracking algorithm implemented by the gaze tracking device based on the updated pose of the frame.

12. The head mounted wearable device of claim 11, wherein the plurality of capacitive sensors includes at least one of: at least one capacitive sensor provided in the first arm portion; at least one capacitive sensor provided in the second arm portion; at least one capacitive sensor provided in upper orbital portions of rim portions of the front frame portion; at least one capacitive sensor provided in lower orbital portions of the rim portions of the front frame portion.

13. The head mounted wearable device of claim 12, wherein: at least one of the at least one capacitive sensor provided in the first arm portion or the at least one capacitive sensor provided in the second arm portion maintains contact with the head of the user in a plurality of different poses of the frame relative to the head of the user, and / or at least one of the at least one capacitive sensor provided in the upper orbital portions or the at least one capacitive sensor provided in lower orbital portions of the rim portions ofthe front frame portion remain spaced apart from the head of the user in the plurality of different poses of the frame relative to the head of the user.

14. The head mounted wearable device of any of claims 11 to 13, wherein the instructions cause the at least one processor to: detect updated data received from each of the plurality of capacitive sensors via the respective plurality of channels; compare the updated data to previous data received from each of the plurality of capacitive sensors via the respective plurality of channels; detect changes in the data received from at least one of the plurality of capacitive sensors based on the comparing; determine an updated pattern defined by the updated data received and the changes; and determine an updated pose of the frame relative to the head of the user based on the updated pattern.

15. The head mounted wearable device of claim 14, wherein the instructions cause the at least one processor to: set the previous data as a baseline value for each of the plurality of channels; and detect the changes based on changes detected from the baseline value for each of the plurality of channels.

16. The head mounted wearable device of any of claims 11 to 15, wherein the instructions cause the at least one processor to adjust the gaze tracking algorithm implemented by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for greater than or equal to a set period of time.

17. The head mounted wearable device of any of claims 11 to 16, wherein the instructions cause the at least one processor to maintain implementation of a current gaze tracking algorithm by the gaze tracking device in response to determining that the updated pose of the frame relative to the head of the user is sustained for less than a set period of time.

18. The head mounted wearable device of claim any of claims 11 to 17, wherein the instructions cause the at least one processor to: determine that the frame remains within a line of sight of the user in the updated pose of the frame relative to the head of the user; and implement a variant of the gaze tracking algorithm that compensates for a change in position of the gaze tracking device relative to the head of the user in the updated pose of the frame, to maintain accuracy of eye tracking performed by the gaze tracking device in the updated pose of the frame.

19. The head mounted wearable device of claim 18, wherein the instructions cause the at least one processor to: determine that the frame is outside of the line of sight of the user in the updated pose of the frame relative to the head of the user, and that the data received from at least one of the plurality of capacitive sensors via at least one of the plurality of channels indicates contact with the user; and operate the head mounted wearable device in a standby mode of operation and temporarily pause operation of the head mounted wearable device.

20. The head mounted wearable device of claim 19, wherein the instructions cause the at least one processor to: determine that the frame is within the line of sight of the user based on the data received from the plurality of capacitive sensors via the plurality of channels, with the head mounted wearable device in the standby mode, terminate operation of the head mounted wearable device in the standby mode and resume normal operation of the head mounted wearable device for the user; detect changes in the data received from the plurality of capacitive sensors via the plurality of channels; determine a new updated pose of the frame relative to the head of the user in response to detecting the changes; and implement a variant of the gaze tracking algorithm by the gaze tracking device based on the new updated pose of the frame.

21. The head mounted wearable device of claim 20, wherein the instructions cause the at least one processor to:detect that at least one of the plurality of capacitive sensors has maintained contact with the head of the user since initiation of a current session and authentication of the user for the current session; determine that the head mounted wearable device has not been removed from the head of the user during the current session; and terminate operation of the head mounted wearable device in the standby mode and resume normal operation of the head mounted wearable device in the current session without re-authentication of the user.

Images