Head mounted display with tactile output

Abstract

The present disclosure relates to a head-mounted display with tactile output. A head-mounted display is disclosed herein, the head-mounted display comprising a display, a tactile output device, and a controller. A display unit is configured to be worn on a user's head and comprises a display for providing a graphical output to the user. The tactile output device is coupled to the display unit. The controller processes a directional input, determines a directional tactile output based on the directional input, and controls the tactile output device to provide the directional tactile output.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] none. Technical Field

[0003] The present disclosure relates to head mounted displays, and in particular to outputs of head mounted displays. Background Art

[0004] A head mounted display is worn on a user's head and provides graphical content, and may also provide auditory content to its user. The graphical content may include graphics for conventional movies, a virtual environment for a game or simulator (discussed below), a productivity program (e.g., a word processor or spreadsheet), etc. The auditory content includes sounds, such as those that appear in a movie, virtual environment, or productivity program. Summary of the invention

[0005] In a specific implementation, the head mounted display includes a display, a tactile output device, and a controller. The display unit is configured to be worn on the head of a user and includes a display for providing a graphical output to the user. The tactile output device is coupled to the display unit. The controller processes a directional input, determines a directional tactile output according to the directional input, and controls the tactile output device to provide the directional tactile output.

[0006] The haptic output device may include a left haptic device and a right haptic device, wherein the directional haptic output includes a left haptic output and a right haptic output. The controller may control the left haptic device to provide a left haptic output, and control the right haptic device to provide a right haptic output.

[0007] The directional input may include one or more of navigation instructions, environmental features of interest, or virtual outputs of interest, where the environmental features of interest are objects in the environment in which the head-mounted display is positioned, and the virtual outputs of interest are at least one of graphical outputs or auditory outputs of the head-mounted display.

[0008] In a specific implementation, the head mounted display includes a display unit, a tactile output device, and a controller. The display unit is configured to be worn on the user's head and includes a display for providing graphical content to the user. The tactile output device is coupled to the display unit. The controller processes a health input, determines a health tactile output based on the health input, and controls the tactile output device to provide a health tactile output. The health input can be a breathing instruction or an exercise instruction for guiding the user.

[0009] In a specific implementation, the head mounted display includes a display unit, a tactile output device and a controller. The display unit is configured to be worn on the user's head and includes a display for providing graphical content to the user. The tactile output device can be removably coupled to the display unit and provide tactile output to the user. The controller controls the tactile output device to provide tactile output. The tactile output device can be removably coupled to the display unit mechanically and electrically. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1A is a front view of a head mounted display on a user's head.

[0011] Figure 1B It is on the user's head Figure 1A Right side view of the head-mounted display.

[0012] Figure 1C It is on the user's head Figure 1A Top view of a head mounted display.

[0013] Figure 1D yes Figure 1A Schematic diagram of a head-mounted display.

[0014] Figure 2 yes Figure 1D A schematic diagram of the controller's hardware configuration is shown.

[0015] Figure 3 is a functional diagram of a head mounted display.

[0016] Figure 4 is a flow chart of a general method for providing tactile output.

[0017] Figure 5 is a flow chart of a method for providing communication information using tactile output with a head mounted display.

[0018] Figure 6 is a flow chart of a method for providing navigation instructions using tactile output of a head mounted display.

[0019] Figure 7 is a flow chart of a method for providing environmental awareness using tactile output of a head mounted display.

[0020] Figure 8 is a flow chart of a method for providing accessibility functionality using tactile output of a head mounted display.

[0021] Fig. 9 is a flow chart of a method for providing breathing instructions using tactile output of a head mounted display.

[0022] Fig.10is a flow chart of a method for providing exercise instructions using tactile output of a head mounted display.

[0023] Fig.11 is a flow chart of a method for adjusting tactile output of a head mounted display. DETAILED DESCRIPTION

[0024] Embodiments of head mounted displays utilizing tactile output, program modules thereof, and methods of using the same are disclosed herein. Tactile output may be provided for a variety of purposes (e.g., to guide a user and / or to attract a user's attention), and may be advantageously provided in some applications independently of graphical content and / or auditory content to provide instructions or other output in a visually and / or auditorily unobtrusive manner.

[0025] See also Figure 1A-Figure 1C , the head mounted display 100 generally includes a display unit 110, a face support 130, and a head support 150. The head mounted display 100 may also be referred to as an HMD, a display system, or a head mounted computer.

[0026] The display unit 110 is configured to be worn on the user's eyes and provide graphical content thereto, which may be a computer-generated reality (as discussed in further detail below). The display unit 110 generally includes a base 112 and one or more displays 114. The base 112 is a main structure, such as a housing, that is configured to be supported on the user's face and to which the one or more displays 114 are coupled. The one or more displays 114 may be hidden by the base 112, as indicated by the displays 114 being shown in phantom.

[0027] One or more displays 114 output graphical content to the user. One or more displays 114 may be provided in any suitable number, such as one display in the display 114 for both eyes of the user, one display in the display 114 for each eye of the user, or multiple displays in the display 114 for each eye. For simplicity, one or more displays 114 are referred to individually (i.e., referred to as "display 114") hereinafter, but it should be understood that the display unit 110 may include more than one display 114. The display 114 may be, for example, a suitable display panel, such as a liquid crystal display panel ("LCD"), a light emitting diode display ("LED"), a micro light emitting diode display ("micro-LED" or "μLED"), or other types of displays. In another example, the display 114 may be configured as a projector and a reflector (e.g., an opaque mirror or a transparent lens) that reflects the projected light back to the user's eyes, or a transparent display.

[0028] The face support 130 engages the user's face for supporting the display unit 110 thereon, wherein the display 114 is in a suitable position for displaying graphical content to the user. The face support 130 may also serve as a light seal to block or prevent ambient light from reaching the user's eyes. The face support 130 is coupled to the display unit 110 and may be formed, for example, of one or more compliant materials that distribute forces to allow the display unit 110 to be comfortably worn by the user. The face support 130 may be removably coupled to the display unit 110, for example, using magnets, clamps, male / female cooperating components, or any other suitable mechanism that allows the user to remove, replace, and / or interchange the face support 130 from the display unit 110. In some embodiments, the face support 130 may include electronic components, in which case the face support is coupled to the display unit 110 both mechanically and electrically (e.g., sends signals to, receives signals from, and / or receives power from).

[0029] The head support 150 engages the user's head for supporting the display unit 110 thereon. The head support 150 may, for example, include a strap that couples to either side (i.e., the left and right sides) of the display unit 110 and extends around the user's head so as to pull the display unit 110 and the face support 130 against the user's face. The head support 150 may be removably coupled to the display unit 110, for example, using magnets, clamps, male / female cooperating components, or any other suitable mechanism that allows the user to remove, replace, and / or interchange the head support 150 from the display unit 110. In some embodiments, the head support 150 may include electronic components, in which case the head support 150 is both mechanically and electrically coupled to the display unit 110 (e.g., sends signals to, receives signals from, and / or receives power from).

[0030] See also Figure 1D In addition to the display 114, the head mounted display 100 may also include various other electronic devices. The other electronic devices include one or more controllers 116 and one or more tactile output devices 118. The other electronic devices may also include one or more audio output devices 120, one or more sensors 122, one or more communication interfaces 124, and / or power electronics 126. As discussed in further detail below, these electronic devices may be provided with various parts of the head mounted display 100 and / or relative to the user, such as provided with the display unit 110, the face support 130, the head support 150, and / or remote therefrom.

[0031] The controller 116 causes the display 114 to provide graphical content, the tactile output device 118 to provide tactile output, and the audio output device 120 to provide audio output. The controller 116 may also control or otherwise communicate with other electronic components of the head mounted display 100, such as the sensor 122 (e.g., controlling and / or receiving sensor signals therefrom) and the communication interface 124 (e.g., controlling and / or sending and receiving communication signals therefrom). The controller 116 executes stored software instructions by which various inputs (e.g., from a user, the sensor 122, and the communication interface 124) are processed for determining and causing outputs of various output devices (e.g., the display 114, the tactile output device 118, and the audio output device 120). The controller 116 may be coupled to the display unit 110 (e.g., coupled to the base 112), the head support 150, or disposed remotely therefrom (e.g., in wired or wireless communication with other electronic components or intervening electronic devices).

[0032] See also Figure 2 , the controller 116 is a computing device capable of implementing the apparatus and methods described herein. In an exemplary hardware configuration, the controller 116 generally includes a processor 216a, a storage device 216b, a memory 216c, a communication interface 216d, and a bus 216e, through which the other hardware components of the controller 116 communicate with each other. The processor 216a can be any suitable processing device capable of executing instructions (e.g., software programming), such as a processor or a central processing unit ("CPU"). The storage device 216b is a long-term non-volatile storage device that stores instructions executed by the processor 216a, such as a hard disk drive ("HDD") or a solid-state drive ("SSD"). The memory 216c is a volatile short-term storage device, such as a random access memory module ("RAM"). The communication interface 216d allows the controller 116 to send signals (e.g., control signals for operating the display 114, the tactile output device 118, and / or the audio output device 120) and receive signals (e.g., from the sensor 122 and / or the communication interface 124). Although shown as a single element, it should be understood that the head mounted display 100 may include more than one controller and / or that various subcomponents of the controller may be disposed in different locations. For simplicity herein, the controller 116 is referred to individually, but it should be understood to include systems having multiple computing devices (e.g., a controller specifically associated with any of the electronic components described herein).

[0033] One or more tactile output devices 118 of the head mounted display 100 provide tactile output to one or more parts of the user's head. Tactile output generally refers to the output of the head mounted display 100 perceived by the user through touch (i.e., via a sense of touch). Specifically, one or more tactile output devices 118 of the head mounted display 100 provide a vibrating tactile output. As discussed in further detail below, such vibrating tactile output can be characterized by frequency, magnitude, position, directionality, and / or duration. The tactile output pattern for a given purpose may also vary over time or other considerations (e.g., position and user sensitivity, among other considerations). A tactile output pattern refers to a combination of frequency, magnitude, position, directionality, and / or duration. The tactile output of a given tactile output module (discussed below) may be different to transmit different information to the user, and the tactile output of different tactile output modules may have different tactile output patterns for the user to distinguish between different information transmitted to it.

[0034] The frequency of a tactile output pattern may refer to the frequency of a continuous output and / or a pulse frequency (e.g., its on / off sequence) of the tactile output device 118. The magnitude of a tactile output pattern is the force applied to a user by the tactile output device 118. The location of a tactile output pattern is the location of the tactile output device 118 relative to the user (e.g., left, right, front, back, top, or other head location). The directionality of a tactile output pattern is the direction of the force perceived by the user. The duration of a tactile output pattern refers to the time that the tactile output pattern is provided. It should be noted that the frequency, magnitude, location, and directionality of a given tactile output pattern may vary over time or based on other considerations.

[0035] In addition, tactile output may be provided alone or in combination with various other outputs. An isolated tactile output may be provided without any other output (e.g., graphics or audio) associated therewith (e.g., based on a common input). An asynchronous combined output may include providing a tactile output and another output (e.g., graphics and / or audio) associated therewith asynchronously (e.g., at different times) therewith (e.g., based on a common input). For example, an asynchronous combined output may start with only tactile output, and over time or as the input changes, add a graphics output and / or an audio output, which may be advantageously used as a navigation tactile output, an environmental tactile output, and / or a virtual awareness tactile output (as discussed below). A simultaneous combined output may include providing a tactile output and another output associated therewith at the same time.

[0036] Each of the haptic output devices 118 may be any type of suitable device capable of outputting a vibratory haptic output, such as a linear resonant actuator (“LRA”), an eccentric rotating mass (“ERM”), a piezoelectric motor or other piezoelectric output device, or a servomotor. It should be noted that not all haptic output devices 118 may be capable of providing different characteristics of a haptic output pattern. For example, a haptic output device 118 may have only one magnitude, one position, and one direction, such that individual haptic output devices 118 may vary only in frequency (e.g., on / off pulses), while the haptic output pattern of multiple haptic output devices of such haptic output devices 118 may also vary in position (e.g., which haptic output device of such haptic output devices 118 is operated).

[0037] Each of haptic output devices 118 may be located in one or more different areas of head mounted display 100 to provide haptic output to one or more different parts of the user's head. Figure 1A , Figure 1B and Figure 1C As shown, in one specific example, the head mounted display 100 includes four haptic output devices 118 at front, left, right, and rear positions, which are schematically depicted. The front haptic device 118F provides haptic output to the front portion of the user's head (e.g., providing front haptic output to the forehead, as shown), the left haptic device 118L provides haptic output to the left portion of the user's head (e.g., providing left haptic output to the left temple, as shown), the right haptic device 118R provides haptic output to the right portion of the user's head (e.g., providing right haptic output to the right temple), and the rear haptic device or back haptic device 118B provides haptic output to the rear portion of the user's head (e.g., providing rear haptic output to the rear of the head, as shown).

[0038] Each of the haptic output devices 118 may directly or indirectly contact a user (e.g., the user's skin or hair) to provide a haptic output thereto. For example, the haptic output device 118 may be covered by another material (e.g., the material of the face support 130, the head support 150, or a covering thereof), through which the user senses the haptic output.

[0039] In other embodiments, haptic output devices 118 may be disposed in fewer, more, and / or different locations. For example, haptic output devices 118 may be disposed in a subset of a front location, a left location, a right location, and a rear location (e.g., only a left location and a right location, only a front location and a rear location, or no rear location). In another example, haptic output devices 118 may alternatively or additionally be disposed in different locations, such as at or along the top of the user's head, cheeks, nose, and / or at or along the sides of the user's head, instead of or in addition to the user's temples.

[0040] The haptic output devices 118 may provide haptic outputs in concentrated areas (e.g., front, left, right, and / or back, as shown). Instead or in addition, the haptic output devices 118 may individually or cooperatively provide haptic outputs over elongated areas. For example, when providing haptic outputs along a location (e.g., the top or side of the head), the haptic outputs may be provided over an elongated area (e.g., extending from front to back over a portion of the user's head).

[0041] Each of haptic output devices 118 may be coupled to display unit 110, face support 130, and / or head support 150. Figures 1A to 1C In the example shown, the front haptic device 118F, the left haptic device 118L, and the right haptic device 118R are coupled to the display unit 110 (eg, providing haptic output via the face support 130 ), while the rear haptic device 118B is coupled to the head support 150 .

[0042] Each of the haptic output devices 118 may be removably or non-removably provided with the head mounted display 100 or a portion thereof coupled with the haptic output device 118 (e.g., provided with the display unit 110, the face support 130, and / or the head support 150). When removably coupled, the haptic output device 118 is intended to be easily removed, replaced, and / or interchanged by a user from the head mounted display 100. For example, the haptic output device 118 may be removably coupled to the head mounted display 100 mechanically and electrically. The haptic output device 118 and the head mounted display 100 (e.g., the display unit 110, the face support 130, and / or the head support 150) may include complementary mechanical mating features and / or mechanisms (e.g., magnets, snaps, clamps, and / or male / female mating structures) to form a mechanical coupling, and may also include complementary electrical mating features (e.g., contacts, spring pins, plugs, and sockets) to form an electrical connection (e.g., to transmit signals and / or power therebetween). A removable haptic output device 118 may be provided as an optional peripheral device that may be optionally coupled to the head mounted display 100. In one example, a detachable haptic output device 118D may be coupled to the head support 150 on an inner surface of the head support (e.g., using snaps 151 that are used to mechanically and electrically couple the detachable haptic output device 118D to the head support 150).

[0043] Where haptic output device 118 is non-removable, haptic output device 118 is configured so as not to be easily removed, replaced, or interchanged by a regular user, but is removable and / or replaceable by a trained technician (e.g., to repair or replace a malfunctioning or non-operating haptic output device 118 ).

[0044] In some embodiments, the head mounted display 100 may include only a non-removable haptic output device 118. In other embodiments, the head mounted display 100 may include a haptic output device that is both non-removable (e.g., in one or more locations on the display unit 110, such as to the forehead at a center location and / or additionally at left and right locations) and removable (e.g., removably coupled to the head support 150 at side and / or back locations).

[0045] Further uses and applications of haptic output device 118 are discussed below.

[0046] The audio output device 120 of the head mounted display 100 provides audio output. The audio output device 120 includes one or more speakers that can be configured as, for example, in-ear headphones, on-ear headphones, or over-ear headphones. The audio output device 120 can be, for example, removably or non-removably coupled to the head support 150 (e.g., Figure 1D ) or any other suitable location (e.g., display unit 110). Alternatively, audio output device 120 may be provided as a separate unit or system (e.g., headphones) that can be used with head mounted display 100 (e.g., in wired or wireless communication therewith) or independently thereof (e.g., with another device).

[0047] The sensors 122 of the head mounted display 100 monitor the environment and / or the user's condition. Those sensors 122 that monitor the environment may include, but are not limited to, one or more outward-facing cameras 122a, one or more depth sensors 122b, one or more ultrasound sensors 122c, one or more position sensors 122d, and one or more audio sensors 122e. Those sensors 122 that monitor the user may include, but are not limited to, one or more inward-facing cameras 122f and one or more physiological sensors 122g. As described below, one or more of the environmental sensors may also monitor the user (e.g., their position and / or their voice) by monitoring the environment. The sensors 122 may be coupled to the display unit 110, such as Figure 1D 1, but may be disposed in any suitable location (eg, head support 150).

[0048] The outward-facing cameras 122a face outward from the head mounted display 100 and monitor the environment around it. The outward-facing cameras 122a may include two forward-facing cameras (e.g., facing forward, such as generally from the perspective of the user's eyes), and may also include one or more side cameras (e.g., facing left and right directions), one or more rear cameras, and / or one or more downward cameras (e.g., facing downward at the front, side, and / or rear of the head mounted display 100). The outward-facing cameras 122a may be any suitable type of camera, including, but not limited to, RGB, grayscale, visible light, and infrared light. The outward-facing cameras 122a, the controller 116, and / or other processing devices may be further capable of processing images captured by the outward-facing cameras 122a, such as with suitable software known in the art, for example, for object recognition and / or tracking.

[0049] One or more depth sensors 122b detect objects in the environment surrounding the head mounted display 100 and / or the distance thereto. For example, one or more depth sensors 122b may be a structured light system (e.g., having a projector and a camera) that projects light in a known pattern, senses light reflected from objects in the environment, and identifies objects and / or the distance thereto using a suitable algorithm. One or more depth sensors 122b may be disposed in any suitable location (e.g., facing forward from the display unit 110). Detection using the depth sensor 122b may be performed in conjunction with the controller 116 or other suitable processing device (e.g., a processing device of the depth sensor 122b itself or associated therewith).

[0050] One or more ultrasonic sensors 122c use ultrasonic signals to detect objects in the environment surrounding the head mounted display 100. The ultrasonic sensor 122c can detect objects that are not otherwise detected by the outward facing camera 122a and / or the depth sensor 122b. The ultrasonic sensor 122c can also detect objects using fewer computing and / or power resources than the outward facing camera 122a and / or the depth sensor 122b. The one or more ultrasonic sensors 122c can be set in any suitable position (e.g., facing forward from the display unit 110 and / or facing backward from the head support 150). Detection using the ultrasonic sensor 122c can be performed in conjunction with the controller 116 or other suitable processing device (e.g., a processing device of the ultrasonic sensor 122c itself or associated therewith).

[0051] One or more position sensors 122d monitor the position, orientation, and / or movement of the head mounted display 100 relative to the environment. In the case where the head mounted display 100 is worn by a user, the position sensor 122d may also and thereby monitor the position, orientation, and / or movement of the user (e.g., their head). The position sensor 122d may include, but is not limited to, a global positioning system sensor, a magnetometer, a gyroscope, an accelerometer, and / or an inertial measurement unit (IMU). Detection using the position sensor 122d may be performed in conjunction with the controller 116 or other suitable processing device (e.g., a processing device of the position sensor 122d itself or associated therewith).

[0052] One or more audio sensors 122e monitor the sounds of the environment, which may include sounds generated by a user (e.g., audio commands). The audio sensor 122e may be, for example, a microphone (mono or stereo). Detection using the audio sensor 122e may be performed in conjunction with the controller 116 or other suitable processing device (e.g., a processing device of the audio sensor 122e itself or associated therewith).

[0053] One or more inward-facing cameras 122f monitor the user. In one example, the inward-facing camera 122f monitors the user's eyes. The inward-facing camera 122f, the controller 116, and / or other processing devices may be further capable of processing images captured by the inward-facing camera 122f, for example, to identify the user, track the user's eye movements, and / or determine the position of the head-mounted display 100 (e.g., its display 114) relative to the user.

[0054] One or more physiological sensors 122g monitor one or more physiological conditions of the user. Such physiological conditions may include, for example, heart rate, perspiration, and temperature, and physiological sensors 122g are of suitable types for monitoring them. Physiological sensors 122g may be disposed in any suitable location, such as display unit 110, face support 130, and / or head support 150. Detection using physiological sensors 122g may be performed in conjunction with controller 116 or other suitable processing device (e.g., a processing device of or associated with physiological sensor 122g itself).

[0055] The power electronics 126 supplies power to various other electronic components of the head mounted display 100. The power electronics 126 may include, for example, one or more batteries. The power electronics 126 may be disposed in any suitable location, such as at the display unit 110 (e.g., Figure 1D ) and / or head support 150. Power delivery from power electronics 126 to other electronic components may be performed in conjunction with controller 116 or other suitable processing device (e.g., a processing device of or associated with the power electronics itself), such as when operating such other electronics (e.g., for output and / or sensing).

[0056] The communication interface 124 sends and / or receives signals from devices external to the head mounted display 100. Such external devices may include a user device 128a, which may act as a user input device for the head mounted display 100. For example, the user device 128a may be a specific control device for the head mounted display 100 (e.g., configured for specific use with the head mounted display 100 or its software). Instead or in addition, the user device 128a may be a multi-purpose device, such as a smart phone, having capabilities and uses independent of the head mounted display 100. The user device 128a may additionally provide additional functionality and / or capabilities to the head mounted display 100. For example, the user device 128a may include a cellular modem that the head mounted display 100 may lack, so that cellular communications may be received by the user device 128a and subsequently transmitted to the head mounted display 100 via its communication interface 124. The external device may be an unassociated device 128b that is not specifically associated with a user or the head mounted device 128. For example, the non-associated device 128b may be another head mounted display 100, a smart phone, or other communication device associated with another person.

[0057] See also Figure 3 and Figure 4 , the head mounted display 100 may include one or more modules 320, according to which tactile output and / or other output is provided. Module 320 processes one or more of inputs 310 and provides one or more outputs 330 according to input 310. As discussed in further detail below, input 310 may include, but is not limited to, signals and / or other information received from and / or generated by sensor 122, communication interface 124, module 320, or other modules in module 320. Module 320 is, for example, a software program that includes instructions executed by controller 116 or other processing device for processing input 310 and providing output 330. Output 330 is provided by tactile output device 118, display 114, audio output device 120, and / or communication interface 124. Input 310, module 320, and output 330 are discussed in more detail below.

[0058] Module 320 may include one or more non-tactile modules 322, which include instructions for providing graphical output, providing audio output, and / or performing other functions but not for providing tactile output. Non-tactile module 322 may, for example, include a movie module for providing graphical content and / or audio content for a movie, a game module for providing graphical content and / or audio content for a game or simulation, or a productivity module for providing graphical content and / or audio for a productivity program (e.g., a word processor, a spreadsheet, etc.).

[0059] Module 320 includes one or more tactile modules 324, which include instructions for providing tactile output according to input, and may also include instructions for providing graphical output, providing audio output, and / or performing other functions. Various combinations of modules 320 can be executed simultaneously, for example, one or more non-tactile modules in non-tactile modules 322 can be executed simultaneously with one or more tactile modules in tactile modules 324. As described in further detail below, tactile module 324 may include one or more of communication module 324A, navigation module 324B, environmental awareness module 324C, accessibility module 324D, breathing module 324E, exercise module 324F, or tactile adjustability module 324G. The functions of each module in module 320 may be included in another module in module 320. Each tactile module and its method in these tactile modules 324 are discussed in more detail below.

[0060] One or more inputs 310 include signals and / or other information received or derived from sensors 122 , communication interface 124 , and / or other modules in modules 320 .

[0061] The sensor 122 serves as input 310 , for example, by sending a sensor signal that is received and processed by the controller 116 or other processing device.

[0062] For example, outward-facing camera 122a, depth sensor 122b, ultrasonic sensor 122c are used as input 310 and for object detection, identification and / or localization. Outward-facing camera 122a, depth sensor 122b and / or ultrasonic sensor 122c can send sensor signals (e.g., image data) that are subsequently processed by controller 116 to perform such object detection, identification, and / or localization as part of haptic module 324 or another module, or can include a separate processing device and send signals with pre-processed information (e.g., detection, identification and / or localization information).

[0063] In another example, the position sensor 122d is used as input 310 and is used to detect the position (e.g., position and / or orientation) of the user. The position sensor 122d may send a sensor signal that is then processed by the controller 116 to determine the position of the user as part of the haptic module 324 or another module, or may include a separate processing device and send a signal with pre-processed information (e.g., the user's position, orientation, and / or movement).

[0064] In another example, the audio sensor 122e is used as input 310 and is used to detect sound events in the environment. The sound event can be, for example, a general sound event (e.g., ambient noise above a certain volume threshold), a user-received sound event (e.g., a voice directed at the user, such as a person saying a user's name), or a user-initiated sound event (e.g., a voice command or other audio gesture provided by the user). The audio sensor 122e may send a sensor signal (e.g., audio data) that is then processed by the controller 116 or other processing device to perform such audio event detection, or may include a separate processing device and send a signal with pre-processed information (e.g., indicating and / or providing details of the audio event, such as a voice command).

[0065] In another example, one or more physiological sensors 122g are used as input 310 and are used to detect a physiological condition of the user (e.g., heart rate, eye movement, and / or galvanic skin response). The physiological sensor 122g sends a sensor signal that is then processed by the controller 116 or other processing device to determine such a physiological condition as part of the haptic module 324 or another module, or may include a separate processing device and send a signal with pre-processed information (e.g., quantified physiological condition).

[0066] Instead or in addition, the communication interface 124 serves as an input 310. For example, the communication interface 124 may receive an external communication signal from a non-associated device 128b directly or indirectly via the user device 128a. The external communication signal may be associated with an incoming video call, a voice call, a video message, a voice message, a written message, or a communication type, for example. In another example, the communication interface 124 receives a user input signal from the user device 128a, such as a user command (e.g., to control a character or object of a game, to make a menu selection). The communication interface 124 sends a corresponding communication signal, which is then processed by the controller 116 or other processing device according to the tactile module 324, or may alternatively include a separate processing device and send a signal with pre-processed information (e.g., to decrypt a message or implement a suitable communication protocol).

[0067] Instead or in addition, the haptic module 324 itself serves as input 310. For example, the haptic module 324 may include instructions by which the haptic output is determined. For example, in a virtual reality environment of a game or simulator module (described below), the haptic module 324 includes instructions by which the haptic output corresponding to the graphical content is provided. In another example, the breathing module 324E may provide breathing instructions at timed intervals.

[0068] Instead or in addition, another module in the module 320 is used as an input 310 to a tactile output module 324, according to which a tactile output of an output 330 is provided. In such cases, another module 320 used as an input 310 may be referred to as a primary output module, and the tactile output module 324 may be referred to as an auxiliary tactile output module. For example, by providing a signal specifically for an auxiliary tactile module to provide a tactile output, the primary output module may be configured to operate (e.g., interact) with the auxiliary tactile output module. Alternatively, the primary output may be configured to operate independently of the auxiliary tactile module (e.g., by not providing a specific signal and without other changes), in which case the auxiliary tactile module processes the output of the primary output module (e.g., its information, graphic content, and / or audio output) to determine and provide a tactile output. It should be noted that both the primary output module and the auxiliary tactile output module may provide output 330 simultaneously, and multiple primary output modules in the primary output module may be used as one or more inputs 310 to the auxiliary tactile module at a given time (i.e., simultaneously) or at different times (i.e., asynchronously).

[0069] As a first non-limiting illustrative example, the primary output module can be a navigation module that provides a primary output of navigation instructions to a user (e.g., visually and / or audibly), while the secondary tactile output module is a secondary navigation module (e.g., that evaluates navigation instructions from any one of a plurality of primary navigation modules) and determines and causes a tactile output based on the primary output (e.g., providing a tactile output pattern with a correct position, thereby tactilely providing navigation instructions to turn right).

[0070] In another non-limiting illustrative example, the primary output module may be a gaming module, which itself may be one of the non-tactile modules 322 or the tactile modules 324, that provides interactive graphical content. The secondary tactile module may be an accessibility module that evaluates graphical content outside the visual field of a visually impaired user and provides tactile output patterns to draw the user's visual attention to the graphical content (e.g., visual events in the periphery of the visual field).

[0071] As will be described below with respect to Figure 5-Figure 11 Discussed in more detail, for a particular tactile module in the tactile module 324, the input 310 may include communication signals, directional inputs, navigation instructions, environmental features of interest, virtual outputs of interest, health inputs, breathing instructions, or exercise instructions, which may be determined individually based on various tactile output modules 324 (or other modules 320) or based on various electronic devices in the electronic devices of the head-mounted display 100 (e.g., sensors 122 and / or communication interface 124).

[0072] Based on input 310, haptic module 324 causes output 330, which includes providing haptic output using haptic output device 118, and may also include providing graphical output using display 114, providing audio output using audio output device 120 and / or providing communication output using communication interface 124. As described above, different haptic output modes can be provided from a single module and between different modules, so that a user can distinguish between information or instructions of such different haptic output modes. Different haptic output modes can differ in characteristics of frequency, magnitude, position, directionality and / or duration, as described above.

[0073] Non-haptic module 322 causes output 330 , which may include providing graphical output using display 114 , providing audio output using audio output device 120 , and / or providing communication output using communication interface 124 , but does not include providing tactile output using tactile output device 118 .

[0074] As will be described below with respect to Figure 5-Figure 11 Discussed in more detail, for a particular haptic module in haptic module 324, haptic output 330 may include communication haptic output, directional haptic output, navigation haptic output, environmental awareness haptic output, virtual environmental awareness haptic output, health haptic output, breathing haptic output, and / or exercise haptic output.

[0075] See also Figure 4 , a high-level method 400 for providing a tactile output may be performed by one or more of the tactile output modules 324. The method 400 generally includes processing 410 an input, determining 420 a tactile output based on the input, and outputting 430 the tactile output.

[0076] Processing 410 of inputs is performed using the controller 116 or a processing device in conjunction with, for example, the sensor 122, the communication interface 124, and / or one of the modules 320 (e.g., the primary graphics module). The inputs may include one or more of the inputs 310 described above (e.g., from the sensor 122, the communication interface 124, and / or from other modules 320). Processing 410 may include, but is not limited to: object detection, identification, and / or location; position determination of a user or head mounted display 100 (e.g., position, orientation, and / or movement); physiological measurements (e.g., measurements of a user's heart rate, eye movement, galvanic skin response, or other physiological conditions); determining and / or evaluating audio events (e.g., identifying and / or evaluating the type, source, and / or content of audio events); communication processing (e.g., determining the type, source, and / or other characteristics of an incoming communication or user command); and / or content evaluation (e.g., analyzing information, images, and / or audio outputted by another module).

[0077] The determination 420 of the tactile output is performed by the controller 116 or other processing device according to the instructions of the module (such as one of the tactile modules 324) and the processing 410 of the input. The tactile output can be, for example, a tactile output pattern determined using a lookup table or algorithm. The tactile output pattern can have the above-mentioned characteristics (e.g., its frequency, magnitude, location, directionality, and / or duration), and can be further determined according to other outputs (e.g., isolated tactile outputs, asynchronous combined outputs, or simultaneous combined outputs as described above).

[0078] The outputting 430 of the haptic output is performed using the haptic output device 114 operated by the controller 116. The controller 116 sends a haptic output signal to the haptic output device 118 for the haptic output device 118 to provide the determined haptic output.

[0079] See also Figure 5-Figure 11 , each tactile module in tactile module 324 and the methods implemented thereby are described in more detail below. As described above, tactile module 324 may include one or more of communication module 324A, navigation module 324B, environmental awareness module 324C, accessibility module 324D, breathing module 324E, exercise module 324F, or tactile adjustability module 324G. Each will be discussed in turn below.

[0080] See also Figure 5 According to the communication module 324A, the head mounted display 100 provides tactile output in response to incoming communication, for example, as part of communication programming or instructions that can be executed simultaneously with providing unrelated graphical content. The tactile output corresponding to the incoming communication is referred to herein as the communication tactile output. The head mounted display 100 receives one or more different communication signals from the non-associated device 128b, where the communication interface 124 is directly from the non-associated device 128b or an intervening infrastructure (e.g., a cellular tower) or indirectly from the user device 128a. The head mounted display 100 then provides the communication tactile output in response to the communication signal. The communication module 324A can be a single communication module that processes the communication signal itself, or the communication module 324A can be an auxiliary tactile output module (as described above) that receives input (e.g., incoming communication information) from one or more other communication modules that can be considered as primary output modules and is otherwise compatible with one or more other communication modules that can be considered as primary output modules (as described above).

[0081] For communication signals of different types and / or sources, the corresponding communication tactile outputs may have different tactile output patterns. Different types of communication signals include, but are not limited to, voice calls, video calls, and written messages (e.g., text messages or emails). Different sources of incoming communications refer to different categories of sources or different individuals, which may be subsets of user-selected sources (e.g., favorite contacts or individual contacts). Different communication tactile outputs have different tactile output patterns that vary by one or more of frequency, magnitude, position, directionality, and / or duration, as described above, to indicate the source and / or type of the incoming communication signal.

[0082] The communication tactile output may be provided as an isolated tactile output, as previously described, in which case no graphical output and / or audio output is provided in response to the communication signal. Therefore and advantageously, the user may not be visually and / or verbally distracted by such graphical indicators and / or audio indicators.

[0083] In one example, the first communication tactile output includes a first tactile output pattern indicating an incoming call (e.g., voice or video), and the second communication tactile output includes a second tactile output pattern indicating an incoming message. The first output pattern and the second output pattern differ in at least one of frequency, magnitude, position, directionality, and / or duration to indicate different types of incoming communications. For example, the first tactile output pattern and the second tactile output pattern differ only in frequency, wherein the first tactile output pattern turns on and off at repeated one-second intervals, and the second tactile output pattern turns on and off at long intervals and short intervals (e.g., two seconds and half a second, respectively). Therefore, a user experiencing the first communication tactile output and the second communication tactile output will be able to distinguish incoming communications as incoming calls or incoming written messages, respectively. Instead or in addition, different tactile output patterns may indicate different sources of communication signals.

[0084] like Figure 5 As shown, method 500 provides a communication tactile output using a head mounted display to indicate an incoming communication. Method 500 may be provided according to instructions of communication module 324A. Method 500 generally includes receiving 510 an incoming communication signal, determining 520 a type and / or source of a first incoming communication signal, determining 530 a communication tactile output according to the type and / or source, providing 540 an output including the communication tactile output, and repeating 550 steps 510, 520, 530, 540 for one or more different incoming communications of different types and / or sources having one or more corresponding different communication tactile outputs. Receiving 510 and determining 520 may be considered as previously relative to Figure 3 and Figure 4 The processing 410 of the input 310 described above can be performed while utilizing another module 320 to provide other outputs independent of the output of the communication module 324A.

[0085] Receiving 510 of incoming communication signals is performed using the communication interface 124 of the head mounted display 100. The communication signals may be received directly from the non-associated device 128b (e.g., from it or from an intervening infrastructure) or indirectly with the user device 128a. Any suitable communication protocol (e.g., a wireless protocol) may be used.

[0086] The determination 520 of the type and / or source of the communication signal is performed using the controller 116 or another processing device (e.g., a processing device of the communication interface 124). For example, the communication signal may include information or otherwise indicate its type and / or source, which may be evaluated by the controller 116 (e.g., according to software programming and / or a lookup table).

[0087] The communication tactile output is determined 530 to have a tactile output pattern based on the source, type, and / or other characteristics of the incoming communication signal, for example, according to software programming and / or a lookup table.

[0088] The providing 540 of output includes providing a communication tactile output and is performed using one or more tactile output devices 118 operated by the controller 116. The communication tactile output may be provided simultaneously with unrelated graphical content provided according to another module in the modules 320 (e.g., in each of watching a movie, playing a game, and using a productivity program). The providing 540 may or may not include providing other outputs, such as graphical output using the display 114 (e.g., graphical output indicating the type and / or source of the incoming communication) and / or auditory output using the audio output device 120 (e.g., a tone or spoken word) that indicates the presence, type, and / or source of the incoming communication.

[0089] Repeat 550 of steps 510, 520, 530, 540 is performed as described above. For different incoming communication signals of different types and / or sources, different communication tactile outputs with different tactile output patterns are determined and provided. Although two different types of incoming communication signals with two different tactile output patterns are discussed above, a greater number of types, sources, or combinations thereof with different tactile output patterns and corresponding communication tactile outputs (e.g., three, four, or more) may be provided.

[0090] See also Figure 6-Figure 8, the head mounted display 100 processes input 310 as directional input and provides output 330 as directional output (including directional tactile output). Directional input is input that has a directional characteristic associated with it, such as navigation instructions (e.g., turning in a certain direction), the location of an object or event in the real environment (e.g., its direction relative to the user or the user's current perspective), or the location of a virtual output of interest (e.g., its direction relative to the user or the user's current perspective). The directional input can be considered to include or be derived from various sensors in the sensors 122 (e.g., environmental sensors 122a-122e) or other input sources. Directional output is output that indicates a direction to the user (e.g., having a directional characteristic associated with it), such as navigation instructions or directional cues to attract the user's attention in different directions. Directional input 310 and directional output 330 are described below with respect to each other. Figure 6 As discussed in the navigation module 324B, environmental objects or events are relative to Figure 7 and Environmental Awareness Module 324C, and the virtual outputs of interest are relative to Figure 8 and Accessibility Module 324D.

[0091] See also Figure 6 According to the navigation module 324B, the head mounted display 100 provides tactile output to provide navigation instructions, for example as part of navigation programming or instructions. The tactile output corresponding to the navigation instructions is referred to herein as the navigation tactile output. The head mounted display 100 determines position information (e.g., position and / or orientation information) and provides navigation tactile output to guide the user in an upcoming manipulation (e.g., a turn or stop) and / or may further provide navigation tactile output to notify the user of approaching such an upcoming manipulation. For example, the navigation tactile output may be provided by the right tactile device 118R to notify the user of an upcoming right turn, or by the front tactile device 118F to notify the user of an upcoming stop. The tactile output pattern of the navigation tactile output may change as the user approaches the manipulation to notify the user of its proximity. For example, when the user approaches a turn or stop, the magnitude and / or frequency of the tactile output pattern may increase (e.g., increase the proportion of the open time). The navigation module 324B may be a single navigation module that determines navigation information by itself, or the navigation module 324B may be an auxiliary tactile output module (as described above) that receives input (e.g., navigation instructions) from a primary output module and is otherwise compatible with one or more other navigation modules that may be considered primary output modules. As described above, navigation instructions or basic sensor information (e.g., of the position sensor 122d) may be considered directional input, and the navigation tactile output may be considered directional tactile output.

[0092] like Figure 6As shown, method 600 provides navigation tactile output using a head mounted display to provide navigation instructions. Method 600 may be provided in accordance with instructions of navigation module 324B. Method 600 generally includes determining 610 location information, determining 620 navigation instructions and / or information for reaching a destination in accordance with the location information, determining 630 output including navigation tactile output in accordance with the navigation instructions and / or information, providing 640 output including navigation tactile output, and repeating 650 steps 610, 620, 630, 640, for example, until the destination is reached. Determining 610 and determining 620 may be considered as previously relative to Figure 3 and Figure 4 The processing 410 of the input 310 described above can be performed while utilizing another module 320 to provide other outputs independent of the output of the navigation module 324B.

[0093] The determination of position information 610 is performed using the controller 116 or other processing device in conjunction with the sensors 122 (e.g., the position sensor 122d and / or the outward-facing camera 122a, the depth sensor 122b, and / or the ultrasonic sensor 122c). For example, the sensor 122 transmits a sensor signal including position information or other information (e.g., an image), which is then processed by the controller 116 to determine the position information. The position information may include the user's position, heading, and / or orientation, which may be derived directly from the position sensor 122d or indirectly (e.g., using object recognition) using the outward-facing camera 122a, the depth sensor 122b, and / or the ultrasonic sensor 122c.

[0094] Determination 620 of navigation instructions and / or information is performed by the controller 116 or other processing device based on the position information. For example, navigation instructions may be determined using appropriate mapping information and / or navigation algorithms. Navigation instructions and / or information include directional instructions for the user (e.g., turn or stop and / or distance to such turn or stop). For example, the position information (e.g., position, orientation, and movement) may be compared to a database (e.g., a map) to determine the navigation instructions and / or compared to objects observed in the environment (e.g., the location of a doorway). The position information and / or navigation instructions may be considered directional input.

[0095] Determination 630 of outputs including navigation tactile outputs is performed by controller 116 or other processing device according to the navigation instructions. For example, the position and / or directionality of the tactile output pattern of the navigation tactile output can be determined according to the direction of the navigation instructions (e.g., the front tactile output device, the right tactile output device, or the left tactile output device 118 corresponds to a navigation instruction to stop, turn right, or turn left, respectively, or the tactile output device 118 can act on the user directionally by being pushed backward, right, or left corresponding to the navigation instruction to stop, turn right, or turn left, respectively). The frequency and / or intensity of the tactile output pattern of the navigation tactile output can be determined according to the proximity of the user to the manipulation and / or the speed of the user. For example, as the user approaches the manipulation, the intensity can increase and / or the on / off frequency can increase (e.g., turn on more). Determination 630 can also include determining other types of outputs, such as graphical navigation instructions and / or auditory navigation instructions.

[0096] The provision 640 of output including navigation tactile output is performed by a tactile output device 118 controlled by a controller 116 or other processing device. Another output may be provided by a display 114 and / or an audio output device 120. Another output may be provided simultaneously with the provision 640 of the navigation tactile output. The navigation tactile output may be considered a directional tactile output.

[0097] A repetition 650 of steps 610, 620, 630, 640 is performed, for example, until the user reaches the destination.

[0098] See also Figure 7According to the environmental awareness module 324C, the head-mounted display 100 provides tactile output to notify the user of objects and / or events in the environment of the head-mounted display 100 (e.g., their occurrence and / or location). Objects and / or events in the environment may be referred to as environmental features of interest. Tactile output corresponding to environmental features of interest is referred to as environmental tactile output hereinafter. The head-mounted display 100 may detect objects in the environment and provide environmental tactile output to notify the user of the existence and / or location of the object. For example, the object may be an obstacle in the environment, such as a trip hazard or an obstacle (e.g., a wall). Instead or in addition, the head-mounted display 100 may detect events in the environment and provide environmental tactile output to notify the user of the occurrence, type and / or location of the event. For example, the event may be a visual event (e.g., a rapid movement or other visual change relative to the environment, such as lighting) and / or an auditory event (e.g., a relatively loud noise relative to the environment). For example, a car accident may be both a visual event (e.g., a rapid deceleration of the vehicle) and an auditory event (e.g., an impact that produces a loud noise). As described above, environmental features of interest are considered directional input or underlying sensor information (e.g., from sensors 122a-122c), while environmental tactile output can be considered, for example, directional tactile output indicating the direction of the environmental feature of interest relative to the user.

[0099] The tactile output pattern of the ambient tactile output may indicate the location and / or proximity of an object relative to the user. The tactile output pattern may indicate the location of an object (e.g., using the tactile output device 118 in different positions) and / or proximity thereto (e.g., the frequency and / or magnitude of changes in the tactile output). For example, by providing graphical content and / or an auditory (e.g., provided in a peripheral device of the display 114 and / or provided by one of the audio output devices 120 on a given side of the user) indicating the presence and / or location of the object or event, other outputs may be provided in conjunction with the ambient tactile output to indicate the object or event.

[0100] Providing ambient tactile output to indicate environmental features of interest can advantageously provide the user with greater awareness of the environment, including those objects and / or events that may be outside the user's field of view (e.g., behind the user) with display 114, outside of hearing range, and / or otherwise not easily noticed by the user with or without display 114 (e.g., in the dark, with transparent objects, and / or to supplement sensing for vision or hearing impairment).

[0101] like Figure 7As shown, method 700 provides environmental tactile output using a head mounted display. Method 700 may be provided according to instructions of environmental awareness module 324C. Method 700 generally includes observing 710 an environment, detecting 720 environmental features of interest (i.e., objects and / or events in the environment), determining 730 an output including environmental tactile output based on the detection, providing 740 an output including environmental tactile output, and repeating 750 steps 710, 720, 730, 740. Observing 710 and detecting 720 may be considered as previously relative to Figure 3 and Figure 4 The processing 410 of the input 310 described above may be performed while utilizing another module 320 to provide other outputs that may be independent of the output of the situational awareness module 324C.

[0102] Observation 710 of the environment is performed using one or more sensors 122, which may include an outward-facing camera 122a, a depth sensor 122b, an ultrasonic sensor 122c, and / or an audio sensor 122e. Observation 710 may alternatively or additionally be performed using a position sensor 122d by comparing the user's position, orientation, and / or movement with known objects (e.g., with a map).

[0103] Detection of an environmental feature of interest (e.g., an object and / or event of the environment) is performed by the controller 116 or other processing device based on the observation 720. For example, the sensor 122 sends a sensor signal including information about the environmental feature of interest (e.g., information about the object and / or event, such as presence or occurrence, identity, location, and / or movement), or a signal including other information (e.g., an image) that is processed by the controller 116 or other processing device to determine information about the environmental feature of interest. The environmental feature of interest (e.g., an object and / or event of the environment) or the underlying sensor information (e.g., sensor information of sensors 122a-122c) can be considered as directional input.

[0104] The determination 730 of the output including the ambient tactile output is performed by the controller 116 or other processing device according to the detection 720 of the environmental feature of interest (e.g., object and / or event information). For example, the position and / or directionality of the tactile output pattern of the ambient tactile output can be determined according to the position of the environmental feature of interest, such as the tactile output pattern including the front tactile output device, the right tactile output device, the left tactile output device, or the rear tactile output device 118 corresponding to the object and / or event is located in front of, on the left, on the right, or behind the user. The frequency and / or intensity of the tactile output pattern can be determined according to the proximity of the user to the manipulation and / or the speed of the user, such as the intensity and / or frequency increases as the user approaches the object and / or event. The ambient tactile output can also vary according to the type or other representation of the object and / or event, such as to indicate that an emergency is distinguished from other types of events. The determination 730 can also include determining other types of outputs, such as graphical outputs and / or auditory outputs indicating the position and / or type or representation of the environmental feature of interest.

[0105] The provision 740 of an output including an ambient tactile output is performed by a tactile output device 118 controlled by a controller 116 or other processing device. Another output may be provided by a display 114 and / or an audio output device 120. Another output may be provided simultaneously with the provision 740 of the tactile output pattern. The ambient tactile output may be considered a directional tactile output.

[0106] The repetition 750 of steps 710 , 720 , 730 , 740 is repeated over time (eg, the method 700 may be performed continuously).

[0107] See also Figure 8 According to the accessibility module 324D, the head mounted display 100 provides tactile output to better adapt the functionality of the head mounted display 100 to users with impaired vision and / or hearing, for example. The accessibility module 324D may be a single module that itself provides other functions (e.g., movies, navigation, productivity, communication, etc.), in which case the accessibility module 324D may include instructions to directly provide tactile output (e.g., corresponding to the other output). Alternatively, the accessibility module 324D may be an auxiliary tactile output module (as described above) that receives input from one or more other modules that may be considered as primary output modules and is otherwise compatible with one or more other modules that may be considered as primary output modules. For example, the accessibility module 324D may receive and process video and / or audio signals from another module 320 for determining tactile output. In either case, the accessibility module 324D may be user selectable (e.g., may be turned off by a user who does not have a corresponding impairment) and / or configurable (e.g., tactile output is provided based on the type of impairment, such as the location of vision and / or hearing loss).

[0108] The accessibility module 324D can be configured to draw the user's attention to graphical outputs and / or auditory outputs that may otherwise be less likely to be noticed by the user due to the user's impairment. Such graphical outputs and / or audio outputs can be collectively referred to as virtual outputs of interest. The tactile output corresponding to the virtual output of interest is hereinafter referred to as a virtual awareness tactile output.

[0109] For example, for a user with a visual impairment that limits their effective visual field (e.g., blindness in one eye or impaired peripheral vision), a graphic of interest (e.g., an object and / or event) may be output by the display 114 in an area that is not easily noticed by the user (e.g., on the side of the blind eye or in the periphery of the visual field). A virtual awareness tactile output is provided to attract the user's attention to the graphic of interest, such as by causing the tactile output mode to indicate the object and / or event of interest to the right with the right tactile device 118R. In another example, for a hearing-impaired user, a sound of interest may be output by the audio output device 120, such as to indicate the position of a voiced character or device in a virtual reality environment of a game or simulation module (e.g., a car approaching from the right). A virtual awareness tactile output is provided to attract the user's attention to the source of the sound of interest, such as by providing a tactile output using the right tactile output device 118R to indicate the sound of interest to the right. A virtual object of interest (e.g., a graphic of interest and / or a sound of interest) may be considered a directional input, and a virtual awareness tactile output may be considered a directional output.

[0110] In addition to indicating the location of an object, event, or sound of interest, the virtual awareness tactile output may also indicate its type or other characteristics. For example, the frequency and / or magnitude of the tactile output pattern may be greater for those objects, events, or sounds that are of greater interest or closer than for those objects, events, or sounds that are of less interest or farther away.

[0111] like Figure 8 As shown, method 800 provides virtual conscious tactile output using a head mounted display. Method 800 may be provided according to instructions of accessibility module 324D. Method 800 generally includes determining 810 a virtual output of interest, determining 820 an output including a virtual conscious tactile output according to the output of interest, providing 830 an output including a virtual conscious tactile output, and repeating 840 steps 810, 820, 830, 840. Determining 810 and determining 820 may be considered as previously relative to Figure 3 and Figure 4 The input 310 is processed 410 .

[0112] Determination 810 of a virtual output of interest is performed using the controller 116 or other processing device, for example, according to instructions of the accessibility module 324D. In the case where the accessibility module 324D is a secondary tactile output module of the primary output module, the accessibility module 324D receives and processes the output (graphic and / or auditory) of the primary output module. For example, the graphical output may be processed according to a suitable image recognition algorithm for determining the output of interest (e.g., detection, identification and / or positioning). In another example, the audio output may be processed according to a suitable audio recognition algorithm for determining the output of interest (e.g., detection, identification and / or positioning). The virtual output of interest may be considered a directional input.

[0113] Determination 820 of outputs including virtual awareness tactile outputs is performed using controller 116 or other processing device, for example, using a lookup table based on the virtual output of interest. A tactile output pattern of the virtual awareness tactile output can be determined to have a location and / or directionality indicating the direction in which the virtual output of interest is located. The tactile output pattern of the virtual awareness tactile output can be determined to have a frequency and / or magnitude (e.g., increasing for greater importance) that indicates the type, proximity, and / or importance of the output of interest.

[0114] Determining 820 may also include determining other outputs, such as graphical outputs and / or auditory outputs that may further indicate the direction, proximity, type, and / or importance of the output of interest. For example, a graphic may be provided at the periphery of the user's effective field of view to indicate that the output of interest appears in the direction of the periphery.

[0115] In those instances where the accessibility module 324D includes direct instructions that may correspond to an output of interest, the determination 810 of the output of interest and the determination 820 may be omitted.

[0116] Providing 830 an output including a virtual awareness tactile output pattern is performed by a tactile output device 118 controlled by a controller 116 or other processing device. Another output may be provided by a display 114 and / or an audio output device 120. Another output may be provided simultaneously with providing 830 of the tactile output pattern. The virtual awareness tactile output may be considered a directional output.

[0117] The repetition 840 of steps 810 , 820 , 830 is repeated over time (eg, the method 800 may be performed continuously).

[0118] Additionally, method 800 may be preceded by determining whether to provide accessibility functionality and / or configuring how to provide accessibility functionality. For example, head mounted display 100 may receive input from a user or other source indicating whether the user may have an impairment and what type of impairment (e.g., vision, hearing, and / or position) the user may have. Alternatively, head mounted display 100 may assess the user's impairment.

[0119] See also Figure 9-10 , the head mounted display 100 processes an input 310 as a health input and provides an output 330 as a health tactile output. A health input is an input that has a personal health aspect associated with it, such as breathing instructions or exercise instructions or underlying sensor information (e.g., from sensors 122a-122g). A health tactile output is an output that has a personal health aspect associated with it, such as a physical output for breathing instructions or exercise instructions. The input 310 and output 330 of the breathing instruction are described below with respect to Fig. 9 and the breathing module 324E, and the input and output of the exercise instructions relative to Fig.10 Discussed with Exercise Module 324F.

[0120] See also Fig. 9 According to the breathing module 324E, the head mounted display 100 provides a tactile output to guide the user to breathe, such as for relaxation purposes, which is referred to as a breathing tactile output hereinafter. The breathing instructions can be provided at predetermined time intervals (e.g., every hour) and / or in response to user behavior or physiological conditions (e.g., if little activity is detected based on movement and / or heart rate measured by the sensor 122). The breathing instructions can be output only as a breathing tactile output instead of a graphical output and / or an audio output so as not to distract the user with graphical content and / or auditory content, or can be provided as a breathing tactile output together with a combination of a graphical output and / or an auditory output. The breathing instructions or basic sensor information (e.g., related to user activity) can be considered as a healthy input, and the breathing tactile output can be considered as a healthy tactile output.

[0121] like Fig. 9 As shown, method 900 provides breathing instructions using a head mounted display to provide a breathing tactile output. Method 900 can be provided according to the instructions of breathing module 324F. Method 900 generally includes determining 910 a breathing instruction, determining 920 an output including a breathing tactile output, providing 930 an output providing a breathing instruction, and repeating 940 steps 910, 920, 930 over time.

[0122] The determination 910 of breathing instructions is performed by the controller 116 or other processing device in conjunction with the timer and / or sensor 122. As described above, the breathing instructions can be determined based on regular intervals, the user's activity, or a combination thereof (e.g., an interval after a period of inactivity by the user). The user's activity can be determined, for example, using a position sensor 122d (e.g., sensing the user's movement) and / or a physiological sensor 122g (e.g., measuring the user's heart rate). The breathing instructions can be considered a health input.

[0123] Determination 920 of a breathing tactile output is performed based on the breathing instruction using the controller 116 or other processing device. For example, the breathing tactile output is determined to be used as a physical prompt or instruction for the user to breathe. The breathing tactile output may include a single predefined tactile output mode (i.e., only one tactile output mode is used to provide breathing instructions), or more than one predefined tactile output mode or variable tactile output mode. The output may or may not include a graphical output and / or an auditory output that also provides visual instructions and / or auditory instructions to the user to breathe.

[0124] Providing 930 a breathing tactile output is performed using a tactile output device 118 operated by a controller 116 or other processing device. Other breathing outputs may also be performed using a display 114 and / or an audio output device 120. The breathing tactile output may be considered a health tactile output.

[0125] The repetition 940 of steps 910, 920, 930 is repeated over time (eg, the method 900 may be performed continuously or during a predetermined time period (eg, waking hours)).

[0126] See also Fig.10According to the exercise module 324F, the head mounted display 100 provides a tactile output to guide the user to perform an exercise, such as weight lifting repetitions, a series of repetitions, or a timed interval exercise (e.g., a full-speed run). The tactile output corresponding to the exercise instruction is referred to as the exercise tactile output hereinafter. The exercise instruction may be provided according to a time interval and / or the user's activity (e.g., determined according to a sensor, which may include an outward-facing camera 122a, a position sensor 122d, and / or a physiological sensor 122g). For example, the head mounted display may determine when the user has completed the exercise and provide an exercise tactile output as an exercise instruction to start a new repetition after a specified interruption time after completing a previous repetition or interval and / or end a repetition or interval after a specified interval time. The exercise instruction may be provided only by touch, so that other graphical and / or auditory content may be provided to the user, and visually and / or auditorily uninterrupted when providing the exercise instruction. For example, the graphical content may be an optical or video pass-through through which the user views the environment. The exercise module 324F may provide the main output of the display 114 or only the graphical output when operating. Exercise instructions or basic sensor information (e.g., related to a user performing an exercise or previous exercise) may be considered a health input, while breathing tactile output may be considered a health tactile output.

[0127] like Fig.10 As shown, method 1000 provides exercise tactile output using a head mounted display to provide exercise instructions to a user. Method 1000 may be provided according to instructions of exercise module 324F. Method 1000 generally includes providing 1010 a visual pass-through of an environment, determining 1020 an exercise instruction, determining 1030 an output including a tactile output to provide the exercise instruction, providing 1040 an output including an exercise tactile output to provide the exercise instruction, and repeating steps 1010, 1020, 1030, 1040 over time.

[0128] The provision 1010 of a visual pass-through of the environment may be performed passively as an optical pass-through or actively as a video pass-through. In the case of an optical pass-through, the user is able to directly observe the environment, such as through a transparent lens. In the case of a video pass-through, the environment is observed using an outward-facing camera 122a, and the video image captured thereby is output by the display 114.

[0129] Determination 1020 of exercise instructions is performed by controller 116 or other processing device in conjunction with timer and / or sensor 122. As described above, the exercise instructions are to perform an exercise (e.g., a repetition, a series of repetitions, or a timed exercise interval). The timing of the exercise instructions may be determined according to regular intervals or intervals from the last exercise completed (e.g., by visually monitoring, measuring movement, and / or measuring the user's physiological condition with sensor 122). The exercise instructions may include type, duration, number of repetitions, or other guidance information. The exercise instructions may be considered health input.

[0130] Determination 1030 of outputs including exercise tactile outputs is performed using controller 116 or other processing device. The tactile output pattern of the exercise tactile output can be determined based on the type, duration, or number of repetitions of the exercise. For example, the tactile output pattern can differ in frequency, magnitude, location, direction, and / or duration based on the type of exercise (e.g., push-ups or sit-ups), duration (e.g., long full-speed sprint vs. short full-speed sprint), or number (e.g., on / off pulses to indicate one vs. more than one repetition).

[0131] Providing 1040 the exercise tactile output is performed using tactile output device 118 operated by controller 116 or other processing means. Other exercise guidance outputs may also be performed using display 114 and / or audio output device 120. The exercise tactile output may be considered a fitness tactile output.

[0132] Repetitions 1050 of steps 1010, 1020, 1030, 1040 are repeated, for example, over one set of repetitions or over a workout comprising a series of different sets of repetitions.

[0133] See also Fig.11 According to the haptic adjustment module 324G, the tactile output provided according to any of the other haptic modules 324 disclosed herein can be adjusted. For example, the user may have thick hair that reduces sensitivity to the tactile output, in which case a stronger (e.g., higher magnitude) tactile output may be warranted. Alternatively, the user may simply prefer a weaker tactile output. In one specific implementation, the user can select the intensity of the tactile output. Instead or in addition, if the user does not respond to the lower intensity tactile output (e.g., does not turn their head in response to the tactile output), the head mounted display can determine the intensity of the tactile output, for example, increasing the intensity of the tactile output. The tactile adjustment module 324G may be disposed within another tactile module in the tactile modules 324, or may be used as an auxiliary tactile output module (as described above), in which case the determined tactile outputs of the other tactile modules 324 (e.g., 324A, 324B, 324C, 324D, 324E, 324F) are used as input 310 to the tactile adjustment module 324G.

[0134] A method 1100 for adjusting intensity of a tactile output is provided. The method 1100 generally includes determining 1110 an intensity of a tactile output, determining 1120 the tactile output based on the intensity, providing 1130 the tactile output, and repeating determining 1120 and providing 1130 based on the determination 1120 of the intensity.

[0135] The determination 1110 of the intensity of the tactile output is performed using the controller 116 or other processing device in conjunction with the sensor 122 and / or the communication interface 124. In one specific implementation, the user specifies the intensity (e.g., relative intensity, such as low intensity, normal intensity, or high intensity) of the tactile output, which can be received by the head-mounted display 100, for example, as a voice command (e.g., using the audio sensor 122e) or as input from the user device 128a (e.g., via the communication interface 124). Instead or in addition, the head-mounted display 100 determines the intensity (e.g., relative intensity) of the tactile output, for example, by varying the intensity of the tactile output until the user responds to the tactile output (e.g., moves their head in unison at a given intensity in close proximity to the tactile output).

[0136] For example, during continued use of one of the tactile output modules 324 or while another tactile output module 324 is in use, repetition 1140 of steps 1120 , 1130 is repeated to provide tactile output, for example, according to intensity.

[0137] Determination 1120 of the tactile output is performed using controller 116 or other processing device according to determination 1120 of the intensity. For example, the tactile output can be determined as described for each of the other tactile modules 324 described previously, wherein its magnitude is reduced or increased relative to a normal level according to the previously determined intensity. For example, the communication tactile output, navigation tactile output, environmental awareness tactile output, virtual awareness tactile output, breathing tactile output, exercise tactile output, directional tactile output, and / or health tactile output determined by other tactile modules 324 can be adjusted up or down according to determination 1110 of the intensity.

[0138] Providing 1130 of tactile output is performed using tactile output device 118 controlled by controller 116 or other processing means.

[0139] See also Figure 3-Figure 11It should be understood that the various tactile output modules 324 can be used in any suitable combination with each other (e.g., their methods are implemented simultaneously with each other). In such cases, their various tactile outputs can have different tactile output modes, so that the user can distinguish between different tactile outputs. For example, the communication module 324A, the navigation module 324B, and the environmental awareness module 324C can be used simultaneously, so that the communication tactile output, the navigation tactile output, and the environmental tactile output all have different tactile output modes. Similarly, the communication module 324A, the environmental awareness module 324C, and the exercise module 324F can be used simultaneously, so that the communication tactile output, the environmental tactile output, and the exercise tactile output all have different tactile output modes.

[0140] The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0141] In contrast, a computer generated reality (CGR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic systems. In CGR, a subset of a person's physical movements, or representations thereof, are tracked, and in response, one or more features of one or more virtual objects simulated in the CGR environment are adjusted in a manner that complies with at least one law of physics. For example, a CGR system may detect a person's head turns, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to features of virtual objects in a CGR environment may be made in response to representations of physical movements (e.g., voice commands).

[0142] A person may sense and / or interact with a CGR object using any of their senses, including vision, hearing, touch, taste, and smell. For example, a person may sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, an audio object may enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, a person may sense and / or interact only with an audio object.

[0143] Examples of CGR include virtual reality and mixed reality.

[0144] A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes a plurality of virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing avatars of people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment, and / or through a simulation of a subset of the person's physical movements within the computer-generated environment.

[0145] In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment designed to include sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtuality continuum, a mixed reality environment is anything between a fully physical environment at one end and a virtual reality environment at the other end, but not including both ends.

[0146] In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. In addition, some electronic systems used to render the MR environment can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment or representations thereof). For example, the system can cause motion so that virtual trees appear stationary relative to the physical ground.

[0147] Examples of mixed reality include augmented reality and augmented virtuality.

[0148] An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on a transparent or translucent display so that a person uses the system to perceive virtual objects superimposed on a physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of a physical environment that are representations of the physical environment. The system combines the image or video with the virtual object and presents the composition on an opaque display. People use the system to indirectly view the physical environment via an image or video of the physical environment and perceive virtual objects superimposed on the physical environment. As used herein, a video of a physical environment displayed on an opaque display is referred to as a "transparent video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting an AR environment on an opaque display. Further alternatively, the system may have a projection system that projects virtual objects into a physical environment, for example as a hologram or on a physical surface, so that a person using the system perceives the virtual objects superimposed on the physical environment.

[0149] An augmented reality environment also refers to a simulated environment in which a representation of a physical environment is transformed by computer-generated sensory information. For example, in providing a pass-through video, the system may transform one or more sensor images to apply a selected perspective (e.g., a viewpoint) that is different from the perspective captured by the imaging sensor. For another example, a representation of a physical environment may be transformed by graphically modifying (e.g., enlarging) portions thereof so that the modified portions may be representative but not true versions of the original captured images. For another example, a representation of a physical environment may be transformed by graphically eliminating portions thereof or blurring portions thereof.

[0150] An augmented virtual (AV) environment refers to a simulated environment in which a virtual or computer-generated environment is combined with one or more sensory inputs from a physical environment. The sensory input may be a representation of one or more characteristics of the physical environment. For example, an AV park may have virtual trees and virtual buildings, but the faces of people are realistically reproduced from images taken of physical people. For another example, a virtual object may take the shape or color of a physical object imaged by one or more imaging sensors. For another example, a virtual object may take a shadow that conforms to the position of the sun in the physical environment.

[0151] There are many different types of electronic systems that enable people to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on people's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smart phones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smart phone). A head-mounted system can incorporate one or more imaging sensors for capturing images or videos of a physical environment, and / or one or more microphones for capturing audio of a physical environment. Instead of an opaque display, a head-mounted system can have a transparent or translucent display. A transparent or translucent display can have a medium through which light representing an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, silicon-based liquid crystal, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects a graphic image onto a person's retina. The projection system can also be configured to project a virtual object into a physical environment, such as a hologram or onto a physical surface.

[0152] As described above, one aspect of the technology of the present invention is to collect and use data from various sources to provide tactile output to the user. The present disclosure contemplates that in some instances, these collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0153] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit the user. For example, the personal information data can be used to provide tactile output such as for accessibility. In addition, the present disclosure also anticipates other uses of the personal information data that benefit the user. For example, health and fitness data can be used to provide insights into the user's overall health, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0154] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

[0155] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of providing tactile output, the technology of the present invention may be configured to allow users to choose to "opt in" or "opt out" to participate in the collection of personal information data during registration for the service or at any time thereafter. In another example, a user may choose not to provide accessibility information or other personal information for providing tactile output. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0156] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0157] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, tactile output may be selected and delivered to a user based on non-personal information data or a small amount of personal information, such as the content of a device request associated with the user, other non-personal information, or publicly available information.

Claims

1. A head mounted display, comprising: a display unit configured to be worn on a user's head and having a display for providing graphical output to the user; a head support coupled to the display unit; Haptic output devices, including: a front haptic output device configured to contact a forward portion of the head of the user and coupled to the display unit; and a rear haptic output device configured to contact a rear portion of the head of the user and coupled to the head support; and a controller that processes a directional input, determines a directional tactile output based on the directional input, and controls the tactile output device to provide the directional tactile output to the front tactile output device or the rear tactile output device, The directional input includes a navigation instruction, and the controller controls the tactile output device to provide the navigation instruction to the user by directionally acting on the user based on the navigation instruction, so that the user senses a force from at least one of the tactile output devices acting on the head of the user in a direction corresponding to the navigation instruction.

2. The head mounted display of claim 1, wherein the tactile output device comprises: a left tactile output device configured to provide feedback to a left portion of the head of the user and coupled to the display unit; and a right tactile output device configured to provide feedback to a right portion of the head of the user and coupled to the display unit, and The controller controls the left tactile output device, the right tactile output device, the front tactile output device, and the rear tactile output device to provide the navigation instruction to the user.

3. The head mounted display according to claim 2, wherein the controller controls the rear tactile output device to provide the navigation instruction by directionally acting on the rear portion of the head of the user, so that the user senses the force in a left direction or a right direction from the rear tactile output device acting on the head of the user.

4. The head mounted display according to claim 3, wherein the controller controls the front tactile output device to provide the navigation instruction by directionally acting on the forward part of the head of the user, so that the user senses the force in the left direction or the right direction from the front tactile output device acting on the head of the user.

5. The head mounted display of claim 1, further comprising a position sensor configured to monitor a position of the head mounted display, wherein the controller is configured to determine the navigation instructions based on the position of the head mounted display.

6. A head-mounted display according to any one of claims 1 to 4, wherein the directional input includes an environmental feature of interest, the environmental feature of interest being an object in the environment in which the head-mounted display is positioned, and the navigation instructions indicate the direction of the environmental feature of interest relative to the user.

7. The head-mounted display according to claim 6 further comprises an environmental sensor, wherein the environmental sensor is configured to sense the object in the environment in which the head-mounted display is positioned, and the controller detects the environmental feature of interest based on the environmental sensor sensing the object.

8. A head-mounted display according to any one of claims 1 to 4, wherein the directional input includes a virtual output of interest, the virtual output of interest is at least one of the graphical output or the auditory output of the head-mounted display, and the navigation instructions indicate the direction of the virtual output of interest within the virtual environment.

9. The head mounted display of claim 8, wherein the at least one of the graphical output or the auditory output includes the virtual environment, and the controller processes the graphical output or the auditory output to determine the virtual output of interest within the virtual output.

10. The head mounted display according to any one of claims 1 to 4, wherein the haptic output device is removably coupleable mechanically and electrically to the display unit.

11. The head mounted display of claim 10, further comprising a head support coupled to the display unit and surrounding the user's head to support the display unit on the user's head, wherein the haptic output device is mechanically and electrically removably coupleable to the head support.

12. A head mounted display, comprising: a display unit configured to be worn on a user's head and having a display for providing graphical content to the user; one or more sensors configured to monitor movement of the user; a tactile output device coupled to the display unit and configured to provide a tactile output pattern to the head of the user according to a healthy tactile output, wherein the tactile output pattern includes a magnitude of the tactile output pattern, a directionality of the tactile output pattern, and a duration of the tactile output pattern; and a controller that determines an exercise instruction based on the motion of the user monitored by the one or more sensors, determines the healthy tactile output according to the exercise instruction, and controls the tactile output device to provide the tactile output pattern to the head of the user according to the healthy tactile output, Wherein the controller is configured to change at least one of the magnitude, the directionality, and the duration of the tactile output pattern based on the movement of the user monitored by the one or more sensors.

13. The head mounted display of claim 12, wherein the controller determines breathing instructions provided in response to the movement of the user monitored by the one or more sensors, and wherein the controller determines the healthy tactile output based on the breathing instructions.

14. A head-mounted display according to claim 12, wherein the exercise instructions include instructions to perform at least one of a repetition, a series of repetitions, or a timed interval exercise, and wherein the controller is configured to change at least one of the magnitude, the directionality, or the duration of the tactile output pattern based on the repetition, the series of repetitions, or the timed interval exercise.

15. The head mounted display of claim 14, wherein the exercise instructions include instructions to perform at least one of the repetitions, the series of repetitions, or the timed interval exercise at predetermined time intervals measured from completion of a previous exercise.

16. The head mounted display of claim 15, wherein the completion of the exercise is determined based on the movement of the user monitored by the one or more sensors.

17. The head mounted display of any one of claims 13 to 16, wherein the haptic output device is mechanically and electrically removably coupleable to the display unit.

18. The head mounted display of claim 17, further comprising a head support coupled to the display unit and surrounding the user's head to support the display unit on the user's head, wherein the haptic output device is mechanically and electrically removably coupleable to the head support.

19. A head mounted display, comprising: a display unit configured to be worn on a user's head and having a display for providing graphical content to the user, the display unit including mechanical mating features and electrical mating features; a haptic output device that provides a tactile output to the head of the user, the haptic output device comprising complementary mechanical mating features and complementary electrical mating features; and a controller that controls the tactile output device to provide the tactile output, wherein the mechanical mating feature of the haptic output device is removably coupled to the complementary mechanical mating feature of the display unit to mechanically couple the haptic output device to the display unit, and Wherein the electrical mating feature of the haptic output device is removably couplable to the complementary electrical mating feature of the display unit to electrically couple the haptic output device to the display unit and the controller.

20. The head mounted display of claim 19, wherein the tactile output device is a first tactile output device, the mechanical mating feature is a first mechanical mating feature, the electrical mating feature is a first electrical mating feature, the complementary mechanical mating feature is a first complementary mechanical mating feature, and the complementary electrical mating feature is a first complementary electrical mating feature, the head mounted display further comprising: a second tactile output device comprising a second mechanical mating feature and a second electrical mating feature; as well as a head support coupled to the display unit and surrounding the head of the user to support the display unit on the head of the user, the head support including a second complementary mechanical mating feature and a second complementary electrical mating feature, wherein the second mechanical mating feature of the second haptic output device is removably coupled to the second complementary mechanical mating feature of the head support to mechanically couple the second haptic output device to the head support and the display unit, and Wherein the second electrical mating feature of the second haptic output device is removably coupled to the second complementary electrical mating feature of the head support to electrically couple the second haptic output device to the head support, the display unit, and the controller.

21. The head mounted display of claim 19, further comprising: A face support removably coupled to the display and configured to engage the user's face, wherein the face support covers the haptic output device, and wherein the haptic output device is coupled to the face support via additional complementary mating features on the face support and the haptic output device.

Images