Electronic device with finger sensor

Abstract

The present disclosure relates to electronic devices with finger sensors. An electronic device such as a head-mounted device may have an internal display that displays images to a user through a lens. A head-mounted support structure may be used to support the display and the lens. When the head-mounted device is worn, one or more external displays may be publicly viewable. The head-mounted support structure may have a front face on which the external display is mounted. One or more finger sensors configured to detect touch input, force input, and / or other input from an external object (such as a user's finger) may be included in the head-mounted device. The finger sensor may have an elongated shape extending along the peripheral edge of the external display on the front face. The finger sensor may also be located on other parts of the support structure.

Description

[0001] This patent application claims priority to U.S. Patent Application No. 16 / 919,007 filed on July 1, 2020 and U.S. Provisional Patent Application No. 62 / 904,562 filed on September 23, 2019, which are hereby incorporated by reference in their entirety. Technical Field

[0002] The present disclosure relates generally to electronic devices, and more particularly to electronic devices having input-output components. Background Art

[0003] Electronic devices sometimes include optical components. For example, wearable electronic devices such as head-mounted devices may include a display for displaying images. Input devices such as buttons may also be included.

[0004] Providing output and acquiring input in wearable electronic devices such as head mounted devices can present challenges. If care is not taken, input-output devices can be cumbersome to use and may not provide the desired output. Summary of the invention

[0005] An electronic device such as a head-mounted device may have an internal display that displays images to a user through a lens. A head-mounted support structure may be used to support the display and the lens. One or more external displays located on the head-mounted support structure may be publicly viewable when the head-mounted device is worn.

[0006] The head mounted support structure may have a front face. One or more finger sensors configured to detect touch input, force input, and / or other input from an external object (such as a user's finger) may be included in the head mounted device. The finger sensors may have an elongated shape extending along a peripheral edge of an external display mounted on the front face. Finger sensors may also be located on other portions of the head mounted support structure.

[0007] The finger sensor may have: a touch sensor configured to collect touch input from a finger or other external object, a force sensor configured to collect force input from a finger or other external object, a proximity sensor configured to detect a finger that is not touching the finger sensor, and / or other sensing circuits. A tactile output device in an electronic device may provide tactile feedback (e.g., when a user provides input to the finger sensor).

[0008] The display may overlap the finger sensor. For example, the elongated finger sensor may be overlapped by a corresponding elongated display or other display structure. Icons and other reconfigurable visual elements may be displayed on the elongated display (e.g., to implement a reconfigurable function bar of a wearable electronic device). In some configurations, visual elements such as icons may be moved between a first display such as a rectangular main outer display and a second display such as an elongated strip display that extends along the peripheral edge of the main display and overlaps the elongated strip finger sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a top view of an illustrative electronic device such as a head-mounted device according to one embodiment.

[0010] Figure 2 is a side view of an illustrative electronic device having input-output components according to an embodiment.

[0011] Figure 3 is a front view of an illustrative electronic device having an area for a front-facing display and an elongated sensor, such as a peripheral finger sensor strip, according to an embodiment.

[0012] Figure 4 is a top view of an illustrative electronic device such as with a finger sensor according to an embodiment.

[0013] Figure 5 and Figure 6 is a front view of an illustrative electronic device having a display according to an embodiment.

[0014] Figure 7 is a front view of an illustrative head mounted device with a finger sensor for adjusting device settings, such as lens position settings, in accordance with an embodiment.

[0015] Figure 8 is a perspective view of an illustrative finger sensor formed on a protruding support structure according to an embodiment.

[0016] Fig. 9 is a front view of an illustrative head mounted device with a peripheral finger sensor according to an embodiment. DETAILED DESCRIPTION

[0017] Electronic devices such as wearable electronic devices may include displays, speakers, tactile output devices, and other output devices for presenting output to users. These electronic devices may also include sensors for collecting environmental measurements and user input. The sensor may include one or more sensors for collecting user input from a user's finger or other external object. These sensors (sometimes referred to as finger sensors in this article) may include capacitive touch sensors, optical touch sensors, resistive touch sensors, and / or other sensors for detecting when a finger or other external object provides touch input, may include force sensors, such as strain gauge sensors and other sensors for detecting applied force, and / or may include other sensors for collecting input, such as proximity sensors that detect when a finger or other object is close to a sensor and / or moves through the air adjacent to the device but has not yet touched the device. Finger sensors can be used to collect single-finger input and multi-finger input.

[0018] Figure 1 A top view of an exemplary head mounted device is shown in FIG. Figure 1 As shown, a head-mounted device such as electronic device 10 may have a head-mounted support structure such as housing 12. Housing 12 may include a portion (e.g., support structure 12T) for allowing device 10 to be worn on a user's head. A main housing portion (e.g., support structure 12M) and associated internal housing portions (e.g., internal support structure 12I) may support a display, lenses, and other optical components (e.g., structure 12I may serve as a lens support structure).

[0019] The front side F of the housing 12 may face outward away from the user's head. The back side R of the housing 12 may face the user. During operation, the user's eyes are placed in the eye box 18. When the user's eyes are in the eye box 18, the user can view the content being displayed by the display 14 through the associated lens 22. The display 14 faces inward toward the eye box 18, and therefore may sometimes be referred to as a rear-facing display, an inner display, an inward-facing display, a display that is not publicly viewable, or a private display. The front side F of the device 10 faces away from the eye box 18 and away from the lens 22.

[0020] In some configurations, the optical components, such as display 14 and lens 22, are configured to display computer-generated content overlaid on a real-world image (e.g., a user may view the real world through the optical components). In other configurations, sometimes described herein as examples, real-world light is blocked (e.g., by an opaque housing wall at front face F of housing 12 and / or other portions of device 10).

[0021] In addition to inward-facing optical components such as inner display 14 and associated lens 22 that allow a user who places his eye in eyebox 18 to view images, device 10 may have one or more displays and / or other light-emitting components (e.g., status indicator lights, illuminated button icons, etc.) located on the outer surface of device 10. For example, device 10 may have one or more external displays (sometimes referred to as outward-facing displays or publicly viewable displays) on front face F, such as display 24. Display 24 may present images visible to people near the user when the user is wearing it and when the user uses device 10 to view images on display 14. Display 24 may also be used to display images that can be viewed by the user on the outside of device 10 when device 10 is not worn (e.g., when device 10 is resting in the user's hand or on a table and not on the user's head). Display 24 may be a touch-sensitive display and / or may be a force-sensitive display (e.g., display 24 or a portion of display 24 may overlap with a finger sensor), or, if desired, display 24 may be insensitive to touch and force input. There may be one or more outward-facing displays, such as display 24 in device 10. The tactile output component may overlap one or more of these outward-facing displays, or may be mounted elsewhere in housing 12 (e.g., to provide tactile output when a user provides finger input such as touch input and / or force input to a portion of the display).

[0022] The support structures of the device 10 may include adjustable components. For example, the support structures 12T and 12M of the housing 12 may include adjustable straps or other structures that can be adjusted to accommodate different head sizes. The support structure 12I may include motor-driven adjustable lens mounts, manually adjustable lens mounts, and other adjustable optical component support structures. The user can adjust the structure 12I to adjust the position of the eyebox 18 to accommodate different user interpupillary distances. For example, in a first configuration, the structure 12I may place lenses and other optical components associated with the user's left and right eyes, respectively, in close proximity to each other so that the eyeboxes 18 are separated from each other by a first distance, and in a second configuration, the structure 12I may be adjusted to place the lenses and other optical components associated with the eyebox 18 in a position where the eyeboxes are separated from each other by a second distance greater than the distance.

[0023] In addition to optical components such as displays 14 and 24, device 10 may also include other electronic components 16. The electronic components of device 10 such as displays and other electronic components 16 may include integrated circuits, discrete components, printed circuits, and other circuits. For example, these components may include control circuits and input-output devices.

[0024] The control circuitry of device 10 may include storage and processing circuitry for controlling the operation of device 10. The control circuitry may include storage devices, such as hard disk drive storage devices, non-volatile memory (e.g., electrically programmable read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random access memory), etc. The processing circuitry in the control circuitry may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, graphics processing units, application-specific integrated circuits, and other integrated circuits. Software code may be stored on a storage device in the control circuitry and run on processing circuitry in the control circuitry to implement control operations for device 10 (e.g., data acquisition operations, operations involving the use of control signals to regulate components of device 10, etc.). The control circuitry in device 10 may include wired communication circuitry and wireless communication circuitry. For example, the control circuitry may include radio frequency transceiver circuitry, such as cellular telephone transceiver circuitry, wireless local area network transceiver circuitry, or the like. transceiver circuitry, millimeter wave transceiver circuitry, and / or other wireless communication circuitry.

[0025] Device 10 may be used in a system of multiple electronic devices. During operation, the communication circuitry of device 10 may be used to support communication between device 10 and other electronic devices in the system. For example, one electronic device may transmit video and / or audio data to device 10 or another electronic device in the system. Electronic devices in the system may communicate over one or more communication networks (e.g., the Internet, a local area network, etc.) using wired communication circuitry and / or wireless communication circuitry. The communication circuitry may be used to allow device 10 to receive data from and / or provide data to external equipment (e.g., a tethered computer, a portable device such as a handheld device or laptop computer, an online computing device such as a remote server or other remote computing device, or other electronic equipment).

[0026] Input-output devices of device 10 (e.g., those in component 16) may be used to allow a user to provide user input to device 10. Input-output devices may also be used to gather information about the operating environment of device 10. Output components in the input-output devices may allow device 10 to provide output to the user, and may be used to communicate with external electronic equipment.

[0027] The input-output devices of the device 10 may include one or more displays, such as an internal display 14 and an external display 24. The external display 24 may be formed by a liquid crystal display, an organic light emitting diode display, a display having an array of crystalline semiconductor light emitting diode dies, or a display based on other pixel types. In some configurations, the display in the device 10 may include left and right display devices (for example, the display 14 may be formed by the following components: left and right components, such as left and right scanning mirror display devices, silicon-based liquid crystal display devices, digital mirror devices or other reflective display devices, left and right display panels based on light emitting diode pixel arrays such as organic light emitting display panels or display devices based on pixel arrays formed by crystalline semiconductor light emitting diode dies, liquid crystal display device panels, and / or other left and right display devices aligned with the user's left and right eyes, respectively). In other configurations, the display 14 may include a single display panel that extends across both eyes or uses other arrangements in which the content is provided with a single pixel array.

[0028] The display of device 10 may be used to display visual content for a user of device 10. For example, content presented on display 14 may include virtual objects and other content provided to the display by control circuitry 12, and may sometimes be referred to as computer-generated content. Images on the display, such as images with computer-generated content, may be displayed without the presence of real-world content, or may be combined with real-world content. In some configurations, the real-world image may be captured by a camera (e.g., a forward-facing camera) such that computer-generated content may be electronically overlaid on portions of the real-world image (e.g., where device 10 is a pair of virtual reality goggles with an opaque display).

[0029] The input-output circuitry of device 10 may include sensors. The sensors may include, for example, three-dimensional sensors (e.g., three-dimensional image sensors such as structured light sensors that emit light beams and use two-dimensional digital image sensors to collect image data for three-dimensional images from the light spots produced when the light beams illuminate a target; binocular three-dimensional image sensors that use two or more cameras in a binocular imaging arrangement to collect three-dimensional images; three-dimensional lidar (light detection and ranging) sensors; three-dimensional radio frequency sensors; or other sensors that collect three-dimensional image data), cameras (e.g., infrared and / or visible digital image sensors), gaze tracking sensors (e.g., gaze tracking systems based on image sensors and (if necessary) based on light sources that emit one or more light beams, such as infrared light sources, where the image sensors are used to track the one or more light beams after they are reflected by the user's eyes), touch sensors, buttons, capacitive proximity sensors, light-based ( optical) proximity sensors, other proximity sensors, force sensors such as strain gauges, capacitive force sensors, resistive force sensors, and / or other force sensors configured to measure force input from a user's finger or other external objects on a display, a track pad or other input surface, sensors such as switch-based contact sensors, gas sensors, pressure sensors, humidity sensors, magnetic sensors, audio sensors (microphones), ambient light sensors, light sensors for user measurements, microphones for collecting voice commands and other audio input, sensors configured to collect information about motion, position and / or orientation (e.g., accelerometers, gyroscopes, compasses, and / or inertial measurement units including all of these sensors or a subset of one or both of these sensors), finger sensors (e.g., two-dimensional capacitive fingerprint sensors, two-dimensional optical fingerprint sensors, etc.), and / or other sensors.

[0030] Sensors and other input devices in the input-output devices of device 10 may be used to collect user input and other information. If desired, device 10 may include tactile output devices (e.g., vibrating components overlapped by a display, a portion of a housing wall, and / or other device structures), light emitting diodes and other light sources, speakers (such as earpieces for producing audio output), and other electronic components for input and output. If desired, device 10 may include circuits for receiving wireless power, circuits for wirelessly transmitting power to other devices, batteries and other energy storage devices (e.g., capacitors), joysticks, buttons, and / or other components.

[0031] Some or all of housing 12 may serve as support structures (e.g., see the portion of housing 12 formed by support structure 12T and the portion of housing 12 formed by support structures 12M and 12I). In configurations where electronic device 10 is a head-mounted device (e.g., a pair of glasses, goggles, a helmet, a hat, etc.), structures 12T and 12M and / or other portions of housing 12 may serve as head-mounted support structures (e.g., structures forming a helmet shell, a headband, the temples of a pair of glasses, goggle shell structures, and / or other head-mounted structures). The head-mounted support structures may be configured to be worn on a user's head during operation of device 10 and may support displays, lenses, sensors, other input-output devices, control circuitry, and / or other components.

[0032] Figure 2 2 is a side view of device 10 in an exemplary configuration, in which device 10 is a head-mounted device having a support structure that allows device 10 to be worn on a user's head 26. The support structure of device 10 may include, for example, structure 12T (e.g., a strap such as a headband that extends horizontally) and / or structure 12T' (e.g., a strap such as an overhead headband that extends from front to back along the middle of the user's head). The housing of device 10 and optional structures such as display 24 may include structures located on a front face F of device 10. The support structure forming the body of device 10 may include flat surfaces and / or curved surfaces, such as curved surfaces that create a smooth transition between a front-facing surface of device 10 and a side-facing surface of device 10 (e.g., along the ears and / or other portions of the sides of head 26 of a user).

[0033] Finger sensors, with or without tactile output devices (e.g., sensors such as touch sensors, force sensors, proximity sensors for detecting finger presence and / or finger mid-air gestures), may be located at any suitable location on device 10, such as location 34 extending along some or all of the horizontally extending head band (structure 12T), location 40 extending along some or all of the surface of the overhead band (such as structure 12T), location 32 along an upward-facing surface on an upper portion of the main housing of device 10 (e.g., along an upper peripheral edge of the main housing of device 10), location 36 along an opposite lower (downward-facing) surface of the periphery of the main housing of device 10, location 38 on a portion of the front of device 10 that faces partially forward and partially sideways from head 26, location 28 at the center of front face F, and / or location 30 at a peripheral edge portion on front face F (and, if desired, at an associated peripheral edge portion of support 12T that extends continuously rearward from front face F). These are exemplary locations for forming finger sensors and / or other sensors in component 16. Sometimes positions such as these will be used as examples to describe adjusting the operation of device 10. If desired, other positions in device 10 (e.g., any exposed surface of device 10 and support structures forming device 10) may be provided with sensors, such as finger sensors.

[0034] Figure 3 1 is a front view of device 10 in an exemplary configuration in which a forward-facing display 24 has been formed over a majority of front face F of housing 12. Sensors such as finger sensors may be formed on front face F along one or more portions of the peripheral edge of housing 12. For example, Figure 3 As shown, the sensor 40 (e.g., a finger sensor) may be formed in an elongated shape, such as a strip (e.g., a bar) extending horizontally across the upper peripheral edge of the housing 12. Figure 3 In the example of FIG. 1 , sensor 40 partially extends across a majority of the width of device 10. In other configurations, sensor 40 may extend across more or less of the width of device 10, and / or may be positioned along the left and / or right peripheral edges of housing 12 and display 24 and / or a lower peripheral edge of housing 12.

[0035] If desired, sensor 40 may be separate from display 24. For example, if display 24 is not touch-sensitive, sensor 40 may be used to collect finger input while optionally using display 24 to provide visual output that changes in response to the collected finger input. In an arrangement where display 24 is touch-sensitive, a user may provide finger input using sensor 40 and display 24. For example, when an on-screen option is displayed on display 24, a user may select the on-screen option on display 24 by touching the on-screen option, and a user may adjust an operating parameter of device 10 by moving a finger along the length of sensor 40 (as an example).

[0036] Figure 4 is a top view of device 10 showing how sensors such as finger sensors (e.g., Figure 4 Sensor 42). An upper edge of housing 12 may be characterized by an upper housing surface that is non-parallel to front side F. In this position, sensor 42 extends along an upper peripheral boundary of front side F of front-facing display 24. When device 10 is worn on the head of a user, the surface of sensor 42 may face upward (e.g., sensor 42 may be characterized by a surface normal that is perpendicular or nearly perpendicular to the surface normal of front-facing display 24). If desired, sensor 42 and / or other finger sensors in device 10 may have an associated display (e.g., an organic light emitting display or other display covering the same area on device 10 as the sensor). The display may include a touch sensor component (e.g., a one-dimensional or two-dimensional capacitive touch sensor, an optical touch sensor, etc.), one or more force sensor components (e.g., to detect entry into the touchscreen), and / or a plurality of force sensor components (e.g., to detect entry into the touchscreen). Figure 4 In the example of the page, a force sensor) or other sensor component is used to input force in the downward direction of the page.

[0037] The control circuitry of device 10 may provide interactive visual elements, such as selectable on-screen options, on an external display such as a display overlaying sensor 42. For example, selectable icons 44 may be adjusted in real time by the control circuitry, and the control circuitry may take corresponding actions in response to a user's finger input selecting a given one of icons 44. For example, a display associated with sensor 42 may display icons 44, and one or more of the displayed icons 44 may correspond to selectable options. Finger input may be provided to sensor 42 during operation. When a finger input corresponding to an icon of a selectable option is received, the control circuitry of device 10 may adjust the operation of device 10 to place device 10 in a given operating mode associated with the selectable option. If desired, sensor 42 and / or other fingerprint sensors of device 10 may be fingerprint sensors or may include fingerprint sensors. In this type of configuration, each of different fingerprints of a user (or different fingerprints from different corresponding users) may be used to trigger device 10 to perform different corresponding tasks. For example, in response to receiving finger input from a first finger, device 10 may enter a first operating mode, and in response to receiving finger input from a second finger different from the first finger, device 10 may enter a second operating mode different from the first operating mode. Fingerprints may be used for authentication, for launching specific applications, operating system functions, or other software, and / or for otherwise providing user input to device 10.

[0038] Tactile feedback (e.g., pulsating vibration) may optionally be provided when an option is selected to help confirm to the user that the input has been received by sensor 42. If desired, the selectable on-screen options may include a reconfigurable (or fixed) sliding input button (see, e.g., illustrative slider button 46). The user may use the button (e.g., a touch screen slider button with a movable indicator representing the current state of the slider button) to provide input for changing an analog value (e.g., playback volume, display brightness, etc.). For example, the user may place finger 48 on slider button 46 and may move finger 48 in direction 50 to increase audio volume or increase display brightness, and move finger 48 in direction 52 to decrease audio volume or decrease display brightness (as examples). A configuration may also be used in which Figure 4 The sensor 42 captures input from the user without using a corresponding display and / or without using an associated tactile output device.

[0039] In some arrangements, the display 24 may be covered with a protective covering layer, such as the display covering layer 12CG of the main housing portion 12M. The layer 12CG may have a curved shape (e.g., a shape that wraps around the front of the user's face) and may be formed of a transparent polymer, glass, or other transparent material. The pixels of the display 24 may be overlapped by the layer 12CG so that the user can view the image on the front side F through the layer 12CG. The display 24 in this type of arrangement may be a flexible display that is curved to conform to the curved inner surface of the layer 12CG. The housing structure 12R may have a portion that forms an upward-facing surface for supporting the sensor 42, and may be configured to wrap around the user's facial features (e.g., the structure 12R may have a recessed portion to accommodate the user's nose). The housing structure 12R may have a soft portion (e.g., foam, fabric, etc.) for forming a cushioning structure adjacent to the user's face. In general, the housing 12 may be formed of any suitable material (e.g., glass, ceramics, metals, polymers, fiber composites such as fiberglass and carbon fiber materials, fabrics, wood and other natural materials, and / or other materials).

[0040] exist Figure 5 In the example of , the display 24 has a first forward-facing portion such as a main forward-facing display 24M and a second portion forming an elongated strip display 24D (formed by the same display substrate and / or a separate display substrate). The display 24 (e.g., display 24M and / or display 24D) may be provided with a finger sensor and, if desired, may be provided with an optional tactile output. For example, the elongated strip display 24D may overlap with an elongated strip finger sensor of the same size or similar size. The display 24M may overlap with a two-dimensional touch sensor or other two-dimensional finger sensor. In some configurations, the display 24 may overlap with a two-dimensional finger sensor. A tactile output device may provide feedback in response to the collected finger input. An arrangement in which a portion of the display 24 does not overlap with any finger sensor component may also be used.

[0041] like Figure 5 As shown, in an arrangement where display 24 senses finger input, a user may provide touch input or other finger input to drag and drop visual elements on display 24 (e.g., icons corresponding to applications or other software on device 10). For example, a user may drag and drop an icon such as icon 58 from display 24M to a location such as location 58' on display 24D, and a user may drag and drop an icon such as icon 60 from display 24D to a location such as location 60' on display 24M. Strip display 24D may be used as a dynamic function bar that includes customized function buttons (e.g., user-selected and / or default icons corresponding to applications, operating system functions, or other device functions).

[0042] exist Figure 6 In the example of FIG. 2 , display 24D includes a plurality of user-selectable options 62 . Figure 6 Display 24D may overlap with a finger sensor (e.g., a sensor for collecting touch input, force input, and / or other finger input) and, optionally, with a tactile output device. A user may provide finger input to select a given option in options 62. Options 62 may correspond to navigation functions that move between various on-screen options 64 and select highlights such as highlight 66. Using this type of input arrangement, a user may move highlight 66 to an on-screen option of interest and may then select one of options 62 to select the highlighted option (e.g., one of options 62 may function as a “select” key). Display 24D may display reconfigurable button labels (e.g., icons) and / or fixed button labels in device 10 may be placed over different finger sensors and / or portions of finger sensors. Figure 6 In the example of FIG. 1 , display 24D (and its associated finger sensor) has an elongated strip shape and extends along the upper peripheral edge of display 24M and housing 12 of device 10. This allows a user to provide finger input without obscuring the visual content on display 24M (e.g., without visually blocking on-screen option 64 from being seen). Using a finger sensor extending along the edge of display 24 (e.g., with Figure 6 24D) may help a user provide finger input to device 10 (e.g., in a configuration where display 24M is a touch-insensitive display that is not overlapped by any finger sensor devices). If desired, arrangements in which display 24 overlaps one or more finger sensors (and, if desired, tactile output devices) may also be used. Display 24D (and its associated finger sensors) may face outward (e.g., a surface normal of display 24D may be parallel to a surface normal of display 24M, and / or display 24D may face upward while display 24M faces horizontally outward). Configurations in which display 24D and its associated finger sensors extend vertically along the right peripheral edge or the left peripheral edge of housing 12 may also be used.

[0043] Figure 7 is a front view of device 10 in an exemplary configuration in which a finger sensor is used to adjust the distance between lenses 22 (e.g., to accommodate the user's interpupillary distance). A finger sensor such as finger sensor 40 may extend along an upper peripheral edge of display 24 and a housing of device 10. In this position, a user may place fingers 48 on the left and right portions of finger sensor 40. To increase the spacing between lenses 22, the user may move fingers 48 away from each other (e.g., by sliding fingers 48 away from each other along the surface of finger sensor 40 in direction 70, as shown in FIG. 1 ). Figure 74. The current horizontal position of each finger 48 may correspond to the associated current position of a corresponding one of the lenses 22. The distance between the lenses 22 (e.g., the center-to-center spacing of the lenses 22) may be reduced by moving the fingers 48 toward each other. An electromagnetic actuator (e.g., a motor, etc.) or other lens positioner may be used to position the lenses 22 in response to the finger input. Because the user's fingers are located near the lenses 22 and because the user's fingers are moved in the direction in which the user desires to move the lenses 22, this type of method for making lens spacing adjustments may be intuitive to the user.

[0044] Generally speaking, any suitable computer-controlled actuator may be controlled by providing input to the device 10 (e.g., using one or more finger sensors). For example, the actuator may be used to tighten and / or loosen a headband, adjust a cushion resting between the housing 12 and the front of the user's face (e.g., to adjust the spacing between the user's eyes and the display 14), and / or adjust other mechanical properties of the device 10. In addition, the user may use finger input on the peripheral finger sensors and / or other finger sensors to adjust other device operating parameters. For example, the user may move a finger back and forth on a horizontally extending elongated finger sensor located on the top edge of the front face F to adjust the stereo balance (left and right audio playback volumes associated with left and right speakers on the support structure 12T mounted on the left and right sides of the user's head, or with left and right speakers in an associated headset or wireless earbud set, respectively), and / or may otherwise use a slider input device implemented using an elongated finger sensor in the housing 12 to adjust audio playback settings and / or other device operating settings. In some configurations, device 10 may be used as a remote control device (eg, such that finger input may be used to adjust stereo balance in an external audio system in addition to or instead of adjusting stereo balance in device 10).

[0045] Figure 88 is a perspective view of an exterior portion of housing 12. Finger sensor 80 has been formed on a protruding portion of housing 12 (e.g., on an annular sidewall surface 82 located on protruding support structure 84). Protruding support structure 84 may have a circular profile, a profile with curved edges and / or straight edges, and / or other suitable shapes. Finger sensor 80 may collect finger input from one or more fingers 48 or other external objects. For example, a user may slide a single finger 48 along the curved peripheral edge of protruding support structure 84 to adjust audio volume, screen brightness, or other adjustable operating parameters. If desired, a user may place two (or more) fingers 48 on the periphery of protruding portion 84 while providing input to the finger sensor. In this way, a user may, for example, provide finger input to finger sensor 80 that simulates the twisting of a rotatable knob (even when protruding support structure 84 is fixed and does not rotate when the user twists his finger).

[0046] If desired, the finger sensor may extend around the perimeter of the front face F of the housing 12. Fig. 9 In an exemplary configuration, a visual element such as an icon 88 is presented to the user. The icon 88 may be displayed on the inward-facing display 14 (e.g., in a peripheral area of ​​the user's field of view) and / or on the outward-facing display 24. The device 10 may have a finger sensor, such as a finger sensor 86 extending along the peripheral edge of the housing 12 and the display 24 (e.g., some or all of the annular borders of the display 24). The sensor 86 may be, for example, a capacitive touch sensor, an optical touch sensor, or other touch-sensitive sensing circuit. When the user desires to move the icon 88 along the periphery of the user's field of view (e.g., when the icon 88 is displayed on the display 14), the user may place the finger 48 on the finger sensor 86, and the icon 88 may be dragged and placed to a new desired position such as position 88' along the edge of the user's field of view. In this way, the user can move notification icons and other visual elements on the display 14 directly to a desired peripheral display position (e.g., to customize the position of displaying these icons along the edge of the user's field of view). During operation of device 10, content being viewed by a user (e.g., a moving image) may be displayed in the central primary portion of display 14, while user-customized visual elements (e.g., icons corresponding to areas for incoming messages, information about the current time, calendar entries, and / or other information) may be displayed at desired user-selected locations.

[0047] Fig. 9 The illustrative example of the present invention involves using a ring-shaped finger sensor (finger sensor 86) extending along some or all of the periphery of the front face F of device 10 to move visual elements (icons, etc.) displayed to the user by device 10 on display 14 (or display 24). If desired, the finger sensor may be located on other parts of housing 12, such as in conjunction with Figure 2For example, a user may use finger sensor 30 to move a visual element displayed on the left side of the user's field of view to the right side of the user's field of view, etc. In general, any of the finger sensor positions described in connection with device 10 (e.g., Figure 2 any finger sensor position, etc.) can be used as an adjustable slider button to display customizable interactive buttons (selectable icons) that can be used to drag and drop or otherwise manipulate content that appears on an external display such as display 24, can be used to move visual elements from one part of display 14 to another, and so on.

[0048] As described above, one aspect of the present technology is to collect and use information such as sensor information. The present disclosure contemplates that, in some cases, data including personal information that uniquely identifies or can be used to contact or locate a specific person may be collected. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, username, password, biometric information, or any other identifying information or personal information.

[0049] The present disclosure recognizes that the use of such personal information in the present technology can be used to benefit the user. For example, the personal information data can be used to deliver targeted content that the user is more interested in. Therefore, the use of such personal information data enables the user to control the content delivered. In addition, the present disclosure also anticipates other uses of personal information data that benefit the user. For example, health and fitness data can be used to provide insights into the user's overall health status, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0050] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be subject to federal and / or state laws such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

[0051] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, the present technology may be configured to allow users to choose to "opt in" or "opt out" of participating in the collection of personal information data at any time during or after registration for a service. As another example, a user may choose not to provide a particular type of user data. As another example, a user may choose to limit the length of time that user-specific data is retained. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application ("app"), and then reminded again just before the personal information data is accessed by the application.

[0052] Furthermore, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. Additionally, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. Where appropriate, de-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or characteristics of stored data (e.g., collecting location data at a city level rather than an address level), controlling how data is stored (e.g., aggregating data across users), and / or other methods.

[0053] Thus, while the present disclosure broadly covers the use of information that may include personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may be implemented without access to personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data.

[0054] Physical environment: The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0055] Computer Generated Reality: A computer generated reality (CGR) environment refers to a fully or partially simulated environment that people perceive and / or interact with via an electronic system. In CGR, a subset of a person's physical movements or a representation thereof is tracked, and in response, one or more features of one or more virtual objects simulated in the CGR environment are adjusted in a manner that complies with at least one physical law. For example, a CGR system can detect a person's head turn, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), the features of virtual objects in a CGR environment can be adjusted in response to a representation of a physical action (e.g., a voice command). People can sense and / or interact with CGR objects using any of their senses (including vision, sound, touch, taste, and smell). For example, a person can sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides a perception of a point audio source in 3D space. As another example, an audio object can enable audio transparency, which selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, a person can sense and / or interact only with an audio object. Examples of CGR include virtual reality and mixed reality.

[0056] Virtual Reality: A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes a plurality of virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing avatars of people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment, and / or through a simulation of a subset of the person's physical movements within the computer-generated environment.

[0057] Mixed reality: In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment designed to include sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtual continuum, a mixed reality environment is anything between a fully physical environment at one end and a virtual reality environment at the other end, but not including both ends. In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. In addition, some electronic systems used to present MR environments can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment, or representations thereof). For example, the system can cause motion so that virtual trees appear stationary relative to the physical ground. Examples of mixed reality include augmented reality and augmented virtuality. Augmented reality: An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation of a physical environment. For example, an electronic system used to present an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system can be configured to present virtual objects on a transparent or translucent display so that people use the system to perceive virtual objects superimposed on a physical environment. Alternatively, the system can have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system combines the image or video with the virtual object and presents the composition on an opaque display. People use the system to indirectly view the physical environment via an image or video of the physical environment and perceive the virtual objects superimposed on the physical environment. As used herein, a video of a physical environment displayed on an opaque display is referred to as "transmitted video", meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting an AR environment on an opaque display. Further alternatively, the system can have a projection system that projects virtual objects into a physical environment, such as a hologram or on a physical surface, so that people use the system to perceive virtual objects superimposed on a physical environment. An augmented reality environment also refers to a simulated environment in which a representation of a physical environment is converted by computer-generated sensory information. For example, in providing pass-through video, the system can transform one or more sensor images to impose a selected perspective (e.g., viewpoint) that is different from the perspective captured by the imaging sensor. For another example, the representation of the physical environment can be transformed by graphically modifying (e.g., enlarging) a portion thereof, such that the modified portion can be a representative but not true version of the original captured image. For another example, the representation of the physical environment can be transformed by graphically eliminating or blurring a portion thereof.Augmented Virtual: An augmented virtual (AV) environment refers to a simulated environment in which a virtual environment or a computer-generated environment incorporates one or more sensory inputs from the physical environment. The sensory input may be a representation of one or more characteristics of the physical environment. For example, an AV park may have virtual trees and virtual buildings, but the faces of people are realistically reproduced from images taken of physical people. For another example, a virtual object may take the shape or color of a physical object imaged by one or more imaging sensors. For another example, a virtual object may take a shadow that conforms to the position of the sun in the physical environment.

[0058] Hardware: There are many different types of electronic systems that enable a person to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system can incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Instead of an opaque display, a head-mounted system can have a transparent or translucent display. A transparent or translucent display can have a medium through which light representing the image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, micro-LED, silicon-based liquid crystal, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects a graphic image onto a person's retina. The projection system can also be configured to project a virtual object into a physical environment, such as a hologram or onto a physical surface.

[0059] According to one embodiment, a head-mounted device is provided, comprising: an internal display; a lens through which the internal display can be viewed from an eyebox; a head-mounted support structure configured to support the internal display and the lens, the head-mounted support structure having a front face facing away from the lens and an opposite back face facing the eyebox; and a finger sensor extending along a peripheral edge of the front face of the head-mounted support structure; and a control circuit configured to adjust visual elements displayed by the internal display based on finger input collected by the finger sensor.

[0060] According to another embodiment, the finger sensor includes an elongated strip touch sensor, the head-mounted device includes a first external display located on the front surface and a second external display separated from the first external display, the second external display is supported by the head-mounted support structure and overlaps with the elongated strip touch sensor.

[0061] According to another embodiment, the finger sensor includes a force sensor.

[0062] According to another embodiment, the finger sensor has an elongated shape and extends at least partially across an upper edge of the front face of the head mounted support structure.

[0063] According to another embodiment, the head mounted device includes a publicly viewable display located on the front face.

[0064] According to another embodiment, the head mounted device includes a publicly viewable touch-insensitive display on the front face having a top edge, and the finger sensor extends along the top edge.

[0065] According to another embodiment, the head mounted support structure has a main portion configured to support the inner display, the head mounted support structure has a head strap portion, and at least some of the finger sensors are located on the head strap portion.

[0066] According to another embodiment, the head mounted support structure has an overhead headband, and the finger sensor is formed on the overhead headband.

[0067] According to another embodiment, the head mounted device includes an elongated strip display extending along a peripheral edge of the front face, and the finger sensor overlaps the elongated strip display.

[0068] According to another embodiment, the finger sensor includes an elongated strip touch sensor.

[0069] According to another embodiment, the finger sensor includes a touch sensor.

[0070] According to another embodiment, the head-mounted device includes a first publicly viewable display located on the front surface overlapping the finger sensor and a second publicly viewable display located on the front surface separate from the first publicly viewable display, and the control circuit is configured to move the icon from the second publicly viewable display to the first publicly viewable display.

[0071] According to another embodiment, the head mounted device includes a first publicly viewable display on the front face overlapping the finger sensor and a second publicly viewable display on the front face that is not sensitive to touch.

[0072] According to another embodiment, the head-mounted device includes an actuator coupled to the lens, and the control circuit is configured to move the lens using the actuator to adjust the lens center to lens center spacing in response to finger input received using the finger sensor.

[0073] According to another embodiment, the head mounted device includes a tactile output device, and the control circuit is configured to use the tactile output device to provide tactile output in response to the finger input from the finger sensor.

[0074] According to one embodiment, a head-mounted device is provided, comprising: an inner display; a lens through which the inner display can be viewed from an eyebox; a head-mounted support structure configured to support the inner display and the lens, the head-mounted support structure having a front face facing away from the lens and an opposite back face facing the eyebox; an outer display located on the front face, the outer display configured to be openly viewed when the inner display is being viewed through the lens, the outer display having a peripheral edge; and a finger sensor extending along the peripheral edge of the outer display and not overlapping the outer display.

[0075] According to another embodiment, the finger sensor comprises an elongated strip sensor located on an upper peripheral edge of the outer display.

[0076] According to another embodiment, the external display is touch-insensitive and the head-mounted device includes a strip display overlapping the elongated strip sensor.

[0077] According to one embodiment, a head-mounted device is provided, which includes: an inner display; a lens through which the inner display can be viewed from an eyebox; a head-mounted support structure, which is configured to support the inner display and the lens, the head-mounted support structure having a front face facing away from the lens, an opposite back face facing the lens, and an upper surface that is not parallel to the front face; an outer display located on the front face, the outer display being configured to be openly viewed when the inner display is being viewed through the lens, the outer display having a peripheral edge; and a sensor located on the upper surface and configured to capture finger input.

[0078] According to another embodiment, the sensor comprises a fingerprint sensor.

[0079] According to another embodiment, the sensor includes an elongated capacitive touch sensor overlapping an additional external display, the additional external display being separate from the external display.

[0080] The foregoing is merely illustrative and various modifications may be made to the embodiments described. The foregoing embodiments may be implemented independently or in any combination.

Claims

1. A head mounted device, comprising: Internal display; a lens through which the internal display can be viewed from the eye box; a head mounted support structure configured to support the inner display and the lens, wherein the head mounted support structure has a front face facing away from the lens and an opposite back face facing the eye box; and a finger sensor extending along a peripheral edge of the front face of the head mounted support structure, wherein a portion of the head mounted support structure separates the finger sensor from the inner display; and A control circuit is configured to adjust a visual element displayed by the inner display based on the finger input acquired by the finger sensor.

2. The head mounted device according to claim 1, wherein the finger sensor comprises an elongated strip touch sensor, The head mounted device according to claim 1 further comprises: a first external display, the first external display being located on the front face; and a second external display, the second external display being separate from the first external display, wherein the second external display is supported by the head-mounted support structure and overlaps the elongated strip-shaped touch sensor.

3. The head mounted device of claim 1, wherein the finger sensor comprises a force sensor.

4. A head-mounted device according to claim 1, wherein the finger sensor has an elongated shape and extends at least partially across an upper edge of the front face of the head-mounted support structure.

5. The head mounted device of claim 1 further comprising a publicly viewable display located on the front face.

6. The head mounted device according to claim 1, further comprising: A publicly viewable touch-insensitive display on the front face has a top edge, wherein the finger sensor extends along the top edge.

7. A head-mounted device according to claim 1, wherein the head-mounted support structure has a main portion configured to support the internal display, wherein the head-mounted support structure has a head strap portion, and wherein at least some of the finger sensors are located on the head strap portion.

8. The head-mounted device of claim 1, wherein the head-mounted support structure has an overhead headband, and wherein the finger sensor is formed on the overhead headband.

9. The head mounted device of claim 1, further comprising an elongated strip display extending along a peripheral edge of the front face, wherein the finger sensor overlaps the elongated strip display.

10. The head mounted device of claim 1, wherein the finger sensor comprises an elongated strip touch sensor.

11. The head mounted device of claim 1, wherein the finger sensor comprises a touch sensor.

12. The head-mounted device of claim 1 further comprising a first publicly viewable display located on the front face that overlaps with the finger sensor and a second publicly viewable display located on the front face that is separate from the first publicly viewable display, wherein the control circuit is configured to move the icon from the second publicly viewable display to the first publicly viewable display.

13. The head mounted device of claim 1 further comprising a first publicly viewable display on the front face overlapping the finger sensor and a second publicly viewable display on the front face that is not touch sensitive.

14. The head-mounted device according to claim 1 further includes an actuator coupled to the lens, wherein the control circuit is configured to move the lens using the actuator to adjust the lens center-to-lens center spacing in response to finger input received using the finger sensor.

15. The head mounted device of claim 1, further comprising a haptic output device, wherein the control circuit is configured to provide a haptic output using the haptic output device in response to the finger input from the finger sensor.

16. A head mounted device comprising: Internal display; a lens through which the internal display can be viewed from the eye box; a head mounted support structure configured to support the inner display and the lens, wherein the head mounted support structure has a front face facing away from the lens and an opposite back face facing the eye box; an outer display on the front face, the outer display configured to be publicly viewed when the inner display is being viewed through the lens, wherein the outer display has a peripheral edge; and A finger sensor extends along a peripheral edge of the outer display and does not overlap the outer display, wherein a portion of the head mounted support structure separates the finger sensor from the inner display.

17. A head-mounted device according to claim 16, wherein the finger sensor includes an elongated strip sensor located on an upper peripheral edge of the external display, and wherein the external display is touch-insensitive, and the head-mounted device also includes a strip display overlapping the elongated strip sensor.

18. A head mounted device comprising: Internal display; a lens through which the internal display can be viewed from the eye box; a head mounted support structure configured to support the inner display, the lens, wherein the head mounted support structure has a front surface facing away from the lens, an opposite back surface facing the lens, and an upper surface that is not parallel to the front surface; an outer display on the front face, the outer display configured to be publicly viewed when the inner display is being viewed through the lens, wherein the outer display has a peripheral edge; and A sensor is located on the upper surface, the sensor being configured to capture finger input, wherein a portion of the head mounted support structure separates the sensor from the inner display.

19. The head mounted device of claim 18, wherein the sensor comprises a fingerprint sensor.

20. The head mounted device of claim 18, wherein the sensor comprises an elongated capacitive touch sensor that overlaps an additional external display, the additional external display being separate from the external display.

Images