Display system having audio output device

The display system integrates removable speaker units with head-mounted display units to adjust audio output based on position, addressing the separation of auditory and visual content in VR systems, enhancing user experience through adaptive audio adjustments and visual indicators.

JP2024167187AActive Publication Date: 2024-12-03APPLE INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024118638
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2018-06-14
Filing Date
2024-07-24
Publication Date
2024-12-03
Estimated Expiration
2038-09-27

AI Technical Summary

Technical Problem

Existing virtual reality systems often separate auditory content from visual content, leading to a disjointed user experience and inefficiencies in audio output adjustment based on the position of audio devices relative to the user.

Method used

A display system with a head-mounted display unit and removable speaker units that adjust audio output based on the position of the speaker units relative to the head-mounted display unit, using sensors and controllers to change volume, equalization, and dynamic range, and provide visual indicators for sound output capabilities.

Benefits of technology

Enhances the user experience by providing synchronized and optimized auditory content that adapts to the position of removable speaker units, ensuring seamless integration with both virtual and mixed reality environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167187000001_ABST
    Figure 2024167187000001_ABST
Patent Text Reader

Abstract

To provide a display system having an audio output device.SOLUTION: A display system 100 includes a head-mounted display unit 110 and a detachable speaker unit 132 . The head-mounted display unit 110 outputs a visual content to a user and provides a visual pass-through of a real environment to the user. The detachable speaker unit 132 is detachably coupleable to the head-mounted display unit 110 for providing an aural content to the user. At least one of the visual content or the aural content is changed according to a position of the detachable speaker unit 132 relative to the head-mounted display unit 110.SELECTED DRAWING: Figure 1A
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 684,826, filed June 14, 2018, the entire disclosure of which is incorporated herein by reference.

[0002] This disclosure relates to display systems and, in particular, to audio output devices therefor. [Background technology]

[0003] Virtual reality systems provide content to a user that may include visual and auditory content. The visual content may be provided by a head-mounted display unit (HMD), while the auditory content is often provided by headphones that are physically separate from the head-mounted display unit. Summary of the Invention

[0004] Disclosed herein are implementations of a display system. In one aspect, the display system includes a head-mounted display unit and a detachable speaker unit. The head-mounted display unit outputs visual content to a user and provides the user with a visual pass-through of a real-world environment. The detachable speaker unit can be detachably coupled to the head-mounted display unit to provide auditory content to the user. At least one of the visual content or the auditory content is altered according to a position of the detachable speaker unit relative to the head-mounted display unit.

[0005] The head mounted display may include built-in speakers. Output of auditory content by one or more of the built-in speakers according to the removable speaker unit or its location. Audio signal processing of the auditory content output by the removable speaker unit may change as the removable speaker unit is moved towards a docking location on the head mounted display unit. The audio signal processing includes one or more of modifying a volume, equalization, or dynamic range of the auditory content output by the removable speaker unit. When the removable speaker unit is within the field of view of the head mounted display unit, the visual content may include a visual indicator in spatial proximity to the removable speaker unit. The visual indicator may indicate to a user the sound output capability of the removable speaker unit.

[0006] In another aspect, a display system includes a head mounted display unit and an audio headset, the head mounted display unit including first and second built-in speakers, and the audio headset including first and second detachable speaker units removably coupleable to the head mounted display unit.

[0007] In another aspect, a method is provided for providing content using a display system having a head mounted display unit, a movable audio output device movable relative to the head mounted display unit, a controller, and a sensor, the method including determining, by a controller having a sensor, a position of the movable audio output device, and providing content using the head mounted display unit according to the position of the movable audio output device. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 2 is a side view of the display system. [Figure 1B] FIG. 1B is a top view of the display system of FIG. 1A. [Figure 1C] FIG. 1B is a schematic diagram of the display system of FIG. 1A. [Figure 1D] FIG. 1B is a schematic diagram of a removable speaker unit of the display system of FIG. 1A. [Figure 2A] 2 is a partial side view of another display system which is a variation of the display system of FIG. 1. [Figure 2B] FIG. 2B is a partial rear view of the display system of FIG. 2A. [Figure 3A] 2 is a partial side view of another display system which is a variation of the display system of FIG. 1. [Figure 3B] FIG. 3B is a partial rear view of the display system of FIG. 3A. [Figure 3C] FIG. 3B is a partial side view of a head mounted display unit of the display system of FIG. 3A. [Figure 3D] FIG. 3B is a partial side view of a removable speaker unit of the display system of FIG. 3A. [Figure 3E] FIG. 3B is a partial side view of the display system of FIG. 3A with hidden features shown in dashed lines. [Figure 4A] FIG. 1 is a first diagram of an audio headset in a first state. [Figure 4B] FIG. 4B is a second view of the audio headset of FIG. 4A in a second state. [Figure 4C] 4B is a side view of a display system that is a variation of the display system of FIG. 1A including the audio headset of FIG. 4A. [Diagram 5] 2 is a flow diagram of a method for outputting sound from the display system of FIG. 1. [Figure 6A] 2 is a plot of the sound output from the display system of FIG. 1. [Figure 6B] 6B is a flow diagram of a method for providing sound output from the display system of FIG. 1 according to the plot of FIG. 6A. [Figure 7A]2 is a plot of the sound output from the display system of FIG. 1. [Figure 7B] 7B is a flow diagram of a method for providing sound output from the display system of FIG. 1 according to the plot of FIG. 7A. [Figure 8A] 2 is a plot of gain versus distance for equalizing the sound output from the display system of FIG. 1. [Figure 8B] 8B is a flow diagram of a method for providing sound output from the display system of FIG. 1 according to the plot of FIG. 8A. [Figure 9A] 2 is a plot of dynamic range versus distance for dynamic range compression of the sound output from the display system of FIG. 1. [Figure 9B] 9B is a flow diagram of a method for providing sound output from the display system of FIG. 1 according to the plot of FIG. 9A. [Figure 10A] FIG. 1 is a diagram of a head mounted display unit having a field of view that includes a real environment and an audio output device. [Figure 10B] 8B is another view of a head mounted display unit having the field of view of FIG. 8A and further displaying a visual indicator. [Figure 10C] FIG. 2 is another diagram of a head mounted display unit having a field of view that does not include an audio output device and further displays another visual indicator. [Figure 10D] FIG. 8C is a flow diagram of a method for providing the visual indicator of FIGS. 8B and 8C. [Figure 11] FIG. 1B is a schematic diagram of a hardware configuration for a controller of the display system of FIG. 1A. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Disclosed herein are embodiments of a display system including a head mounted display unit and an audio output device, such as an audio headset. The audio output device is removable from the head mounted display unit or is otherwise movable relative to the head mounted display unit in various ways described herein. Auditory content provided from the audio output device and / or the built-in audio output device may be output according to the position of the movable audio output device in various ways described herein. Visual content may also be provided according to the position of the movable audio output device in various ways described herein.

[0010] 1A-1C, display system 100 includes head mounted display unit 110 and audio headset 130 that is physically movable relative to head mounted display unit 110, such as detachable therefrom (as shown) or articulated thereto. As described in more detail below, display system 100 may form a mechanical and / or electrical connection between head mounted display unit 110 and audio headset 130, and may provide sound to a user via head mounted display unit 110 itself and audio headset 130, and / or may use head mounted display unit 110 to provide cues and / or feedback to a user related to use of audio headset 130. The term "audio headset 130" refers to an audio output device or set of audio output devices (e.g., a detachable speaker unit 132, as described below) that is held in proximity to one or both ears of a user. In the case of multiple (e.g., two) audio output devices, the audio output devices of audio headset 130 may be separate or separable from each other (e.g., they do not need to be secured to each other with a headband extending between them). Audio headset 130 may also be referred to as headphone(s) or earphone(s).

[0011] The display systems disclosed herein may be configured to provide computer-generated reality (e.g., virtual reality or mixed reality), as described below. The term "virtual" as used generally refers to "computer-generated" content or experiences. For example, content described herein may be referred to as virtual or computer-generated content, such as visual content (e.g., also referred to as virtual content or computer-generated visual content or graphics) and auditory content (e.g., also referred to as virtual content or computer-generated auditory content or audio).

[0012] The head mounted display unit 110 generally includes a support 112 and a display screen 114. The support is configured to be worn on a user's head H, thereby supporting the display screen 114 in a suitable position relative to the user's eyes. As shown, the support is configured as a strap that encircles the user's head H. The display screen 114, such as a smartphone screen or a dedicated screen, provides visual content to the user. If the display screen 114 is a smartphone or other detachable video output device, the head mounted display unit 110 can be considered to be coupleable to the display screen 114.

[0013] The display system 100 may be configured to provide a mixed reality experience by providing a visual pass-through through which the user can see the real environment. As shown, the head mounted display unit 110 utilizes a video pass-through, where one or more cameras 116 of the head mounted display unit 110 capture images of the real environment, which are displayed to the user by the display screen 114. Such a display system 100 may also function to provide a virtual reality experience, where images of the real environment are not displayed by the display screen 114, or where the real environment is otherwise partially or totally blocked from the user's view. As an alternative to a video pass-through, the head mounted display unit 110 may utilize an optical pass-through, where the user views the real environment directly. For example, the head mounted display unit 110 may include a reflector that allows both light from the real environment to pass through to the user's eyes and to reflect light from the display screen 114 to the user's eyes to provide visual content.

[0014] The head mounted display unit 110 may further include a speaker 118 that transmits or outputs auditory content to the user's ear. The speaker 118 is coupled to the support 112, for example, in close proximity to the user's ear E and directs sound to the ear. The speaker 118 is fixedly coupled to the head mounted display unit 110 (e.g., to the support 112 or to the housing of the display screen 114) so ​​that it cannot be easily removed from the head mounted display unit. This is in contrast to the speaker unit 132 of the audio headset 130, which is removably coupled to the head mounted display unit 110 so that it is easily removable from the head mounted display unit. Thus, the speaker 118 may be referred to as a fixed audio output device, fixed speaker, built-in speaker, or HMD speaker, while the speaker unit 132 may be referred to as a removable speaker or a removable speaker unit. The audio headset 130 and the removable speaker unit 132 may also be referred to as audio output devices or movable audio output devices.

[0015] The HMD speakers 118 further allow sounds from the real environment to reach the user's ears. For example, the HMD speakers 118 may be positioned above and / or in front of the user's ears, thereby not preventing sounds from the real environment from reaching the user's ears. In this way, the head mounted display unit 110 can provide a mixed reality experience to the user aurally by simultaneously transmitting auditory content to the user while allowing the user to hear the real environment. Thus, the HMD speakers 118 may also be referred to as additional auditory speakers.

[0016] As described in more detail below, the HMD speakers 118 are further configured to enable the audio headset 130 to provide aural content to the user in conjunction with or to the exclusion of the HMD speakers 118. In some embodiments, the HMD speakers 118 may be omitted, in which case the aural content is provided by the audio headset 130.

[0017] The head mounted display unit may also include various sensors to detect conditions related to the user, the head mounted display unit 110, and / or the audio headset 130. For example, in addition to the camera 116, the head mounted display unit 110 may include a motion sensor 120, such as one or more of an accelerometer, gyroscope, magnetometer, inertial measurement unit (IMU), camera, etc., to measure conditions related to the position and / or orientation of the user's head H and / or the head mounted display unit 110. The camera 116 and / or the motion sensor 120 may also be configured to determine the position, orientation, and / or movement of the audio headset 130 relative thereto, for example using visual recognition of the audio headset 130, to detect, locate, and / or track markers on the removable speaker unit 132, and / or to detect, locate, and / or track beacon signals emitted from the removable speaker unit 132. The head mounted display unit 110 may also include an eye sensor 122, such as a camera, that is used to measure the condition of the user's eyes (e.g., focal length, pupil diameter, etc.).

[0018] The audio headset 130 includes removable speaker units 132 that provide auditory content to the user by transmitting sound to the user's ears E. The removable speaker units 132 include, for example, a right removable speaker unit 132r and a left removable speaker unit 132l that are physically associated with the user's right ear E and left ear E, respectively, to provide the auditory content.

[0019] Each of the removable speaker units 132 generally includes a housing 134 as well as one or more speakers 136 and various types of electronics 138. The housing 134 includes the one or more speakers 136 and electronics 138 and is configured to be connected to and supported by the support 112 of the head mounted display unit 110, receive audio signals therefrom, and / or receive power therefrom. The speakers 136 output sound (i.e., auditory content) to a user. The electronics 138 are configured in cooperation with the speakers 136 to provide sound output and may provide other functions.

[0020] As shown, audio headset 130 may have an over-ear configuration, where housing 134 is configured to completely surround a user's ear, or audio headset 130 may have an on-ear configuration, where housing 134 is configured to rest on a user's ear, or an in-ear configuration, where a portion of housing 134 is disposed within a user's ear E (e.g., partially within the ear canal).

[0021] As shown generally in FIG. 1D , the electronics 138 of the speaker unit 132 may include an amplifier 138a, power electronics 138b, signal electronics 138c, sensing electronics 138d, and / or ambient sound electronics 138e. The amplifier 138a is configured to drive the speaker 136 to output sound. The power electronics 138b is configured to power various other electronics of the removable speaker unit 132, such as the amplifier 138a, the signal electronics 138c, and / or the sensing electronics 138d. The power electronics 138b may provide, for example, wired power and / or locally stored power, such as a battery. The signal electronics 138c is configured to receive and / or transmit signals to and / or from another component, such as the head mounted display unit 110 or the controller 150 described below. The signal electronics 138c may include a communication interface for transmitting and / or receiving signals, such as via a wireless, inductive, or optical connection. Such signals may include, for example, audio signals received to output auditory content from the removable speaker unit 132, and / or sensory signals to provide sensory information (e.g., position) of the removable speaker unit 132. The signal electronics 138c may also provide outputs to assist in determining the position and / or orientation of the removable speaker unit 132 relative to the head mounted display unit 130, for example, by providing a passive or active marker (e.g., a visible or non-visible light source) or a beacon (e.g., emitting another type of detectable and locatable output signal). The sensing electronics 138d are configured to sense various parameters of the removable speaker unit 132, such as the position, orientation, and / or movement (e.g., changes in position and orientation) of the removable speaker unit 132 in real space or relative to another portion of the display system 100, such as the head mounted display unit 110, the support 112, or their associated positions or features.The sensing electronics 138d may include one or more suitable sensors for detecting such position, orientation, and / or movement, such as an accelerometer(s), a gyroscope(s), an inertial measurement unit (IMU), a proximity sensor(s), and / or a camera (e.g., for video recognition).

[0022] The ambient sound electronics 138e function to detect ambient sound (e.g., from the real environment using microphone 138e′), process the detected ambient sound (e.g., with signal processor 138e″), and output the processed sound accordingly to speaker 136 (e.g., with amplifier 138a). The processed sound may provide, for example, acoustic transparency and / or active noise cancellation. With acoustic transparency, ambient sound from the real environment is generally reproduced without significant modification as processed sound, which may be referred to as processed ambient sound, and is output by speaker 136. As a result, for example, the user can still hear the real environment when the removable speaker unit 132 is positioned against the user's ear E, but may otherwise prevent (e.g., by insulation and / or blocking) ambient sounds of the real environment from reaching the user's ear E directly. With active noise cancellation, the ambient sound is processed to provide a processed sound that generally cancels or counteracts the ambient sound (e.g., by being at the same amplitude as the ambient sound, but in opposite phase), which may be referred to as a processed cancellation sound, and is output by the speaker 136.

[0023] It should be noted that speaker electronics 138 (e.g., amplifier 138a, power electronics 138b, signal electronics 138c, sensing electronics 138d, and / or ambient sound electronics 138e), or different electronics performing similar functions, may alternatively or additionally be incorporated into head mounted display unit 110. For example, speaker 136 may have a wired connection to head mounted display unit 110 and / or controller 150, while amplification, powering, sensing, and ambient sound functions are performed by electronics of head mounted display unit 110 and / or controller 150.

[0024] In addition to the head mounted display unit 110 and the audio headset 130, the display system 100 further includes a controller 150. As shown, the controller 150 may be external to the head mounted display unit 110 and the audio headset 130 and is in communication with them, such as by using a wired or wireless connection. The controller 150 is generally configured to process various input information (e.g., signals), such as the position and / or orientation of the user, the head mounted display unit 110, and / or the audio headset 130, process such information, and provide output signals for controlling the head mounted display unit 110 (e.g., to output visual and auditory content) and the audio headset 130 (e.g., to output auditory content). The hardware configuration of the controller 150 is described below with reference to FIG. 9. Although shown as a single external unit separate from head mounted display unit 110 and audio headset 130, controller 150 may instead include multiple such controllers or be subdivided into separate units that function cooperatively as controller 150, and controller 150 may be incorporated into one or more of head mounted display unit 110 and detachable speaker unit 132.

[0025] As described above, head mounted display unit 110 and audio headset 130 may be cooperatively configured to provide mechanical, electrical, and / or data connections between head mounted display unit 110 and audio headset 130, and may cooperatively provide sound to a user and / or use head mounted display unit 110 to provide cues and / or feedback to a user related to use of audio headset 130.

[0026] 2A-4B, each of the removable speaker units 132 of the audio headset 130 is removable from the head mounted display unit 110. When attached to the head mounted display unit 110, each of the removable speaker units 132 is physically supported thereby and can further communicate with and / or receive power from the head mounted display unit 110.

[0027] 2A-2B, the display system 200 includes an audio headset 230 having a detachable speaker unit 232 that is magnetically releasably coupleable to a head mounted display unit 210. The head mounted display unit 210 is configured as the head mounted display unit 110 and further includes one or more magnetic attachments 224 that may be referred to as HMD magnetic attachments. The detachable speaker unit 232 is configured as the detachable speaker unit 132 and includes one or more magnetic attachments 244 that correspond to the HMD magnetic attachments 224 of the head mounted display unit 210. The magnetic attachments 224, 244 of the head mounted display unit 210 and the detachable speaker unit 232 may be referred to as the HMD magnetic attachments 224 and speaker magnetic attachments 244, respectively. The magnetic attachments 224, 244 are configured to be magnetically attracted to one another, with one of the magnetic attachments 224, 244 including a permanent magnet and the other including a ferromagnetic material or another permanent magnet of opposite orientation.

[0028] 2A-2B, the head mounted display unit 210 includes two HMD magnetic attachments 224, one in front of and the other behind the user's ears E. As shown, the two HMD magnetic attachments 224 may be coupled to the support 112 of the head mounted display unit 210, for example, to flanges 212a (e.g., protrusions) that extend downwardly to be positioned in front of and behind the user's ears E. The two HMD magnetic attachments 224 include permanent magnets that may be incorporated within or otherwise coupled to the support 112 of the head mounted display unit 210.

[0029] The removable speaker unit 232, such as the housing 134 or its pad 246, may protrude (e.g., extend inward) from the support 112 (e.g., inward of the flange 212a) and be positioned relative to the user, for example, by engaging the head H around the ear E (e.g., in an over-ear configuration) or engaging the user's ear E (e.g., in an on-ear configuration). As a result, the removable speaker unit 232 can be considered to have an inner portion (e.g., the pad 246 and / or the housing 134) that extends laterally inward (e.g., toward the user) from an outer portion (e.g., formed by the speaker magnetic attachment 244) that is coupled to the head mounted display unit 110. The pad 246 may engage and / or circumscribe the user's ear E (e.g., in an on-ear and / or over-ear configuration).

[0030] Each of the detachable speaker units 232 includes two speaker magnetic attachments 244 that are magnetically coupleable to the two HMD magnetic attachments 224 of the head mounted display unit 210. For example, the two speaker magnetic attachments 244 may extend in front of and behind the user's ears E so as to be magnetically coupleable to the HMD magnetic attachments 224. The speaker magnetic attachments 244 may be elongated, for example, and may extend in front of and behind the housing 134 of the detachable speaker units 232. As shown, the speaker magnetic attachments 244 may be configured as posts that include ferromagnetic material or opposing permanent magnets so as to be attracted to the permanent magnets of the HMD magnetic attachments 224 of the head mounted display unit 210. Alternatively, the HMD magnetic attachments 224 may include ferromagnetic material and the speaker magnetic attachments 244 may include permanent magnets that are attracted thereto.

[0031] The head mounted display unit 210 and the removable speaker unit 232 may also be configured to guide the removable speaker unit 232 to a predetermined position relative to them (e.g., a docking position of the head mounted display unit 110 above the user's ear E). For example, each set of magnetic attachments 224, 244 (e.g., a front and back pair of magnetic attachments 224, 244) provides a magnetic force to guide the speaker magnetic attachment 244 of the removable speaker unit 232 to the corresponding HMD magnetic attachment 224 of the head mounted display unit 210. In addition, the non-corresponding magnetic attachments 224, 244 may also function to repel each other, for example, by having permanent magnets with the same orientation. As a result, the removable speaker unit 232 is pushed away from the non-predetermined position. These attractive and / or repulsive forces may be useful to help the user guide the detachable speaker units 132 to their respective docking locations on the head mounted display unit 210 that are outside the user's field of view by providing tactile feedback to the user.

[0032] Instead of, or in addition to, using magnetic forces to guide the removable speaker unit 232 into position, the head mounted display unit 210 and the removable speaker unit 232 may include mechanically cooperating guide mechanisms. For example, the head mounted display unit 210 may include detents within the support 112 that receive and guide the speaker magnetic attachments 244 of the removable speaker unit 232 into position.

[0033] The removable speaker unit 232 may be configured to pivot about the speaker magnetic attachment 244, e.g., to be biased toward or otherwise housed at the user's ear E. In one embodiment, the speaker magnetic attachment 244 is a cylindrical post that extends forward and rearward (e.g., forming a pivot axis 244a). The cylindrical post rotates along and pivots relative to a surface of the HMD magnetic attachment 224 or support 112 of the head mounted display unit 210. In another embodiment, the housing 134 pivots relative to the speaker magnetic attachment 244 without the speaker magnetic attachment moving (e.g., rotating) relative to the HMD magnetic attachment 224 of the head mounted display unit 210, thereby still allowing the housing 134 and the speaker 136 therein to pivot relative to the head mounted display unit 210.

[0034] The pivot axis 244a of the removable speaker unit 232 is formed, for example, by the speaker magnetic attachment 244. The pivot axis 244a may be located (e.g., horizontally and vertically) at a central position of the removable speaker unit 232, as shown, or may be biased vertically (e.g., toward a top end of the removable speaker unit 232) and / or horizontally (e.g., toward the inside or outside of the removable speaker unit 232). In addition, the pivot axis 244a may be located outside the support 112 of the head mounted display unit 210, as shown.

[0035] The removable speaker unit 232 may be further configured to be biased toward the user's ear E about the pivot axis 244a, such as by a spring or magnet. In the example shown in FIGS. 2A-2B, the removable speaker unit 232 may be biased toward the user's ear E by a magnetic force. The head mounted display unit 210 may include a magnet 228 in one orientation disposed above the user's ear E, while the removable speaker unit 232 includes another magnet 248 in the opposite orientation above the user's ear E, for example, by forming or incorporating an elongated member extending upward from the housing 134. The magnets 228, 248 of the head mounted display unit 210 and the removable speaker unit 232 repel each other to bias the removable speaker unit 232 toward the user's ear E about the pivot axis 244a.

[0036] The head mounted display unit 210 and the removable speaker unit 232 are additionally configured relative to the head mounted display unit 210 to provide power to the removable speaker unit 232. As shown, the magnetic attachments 224, 244 conduct electricity from the head mounted display unit 210 to the removable speaker unit 232 to provide power to the amplifier 138a and other electronics (e.g., the signal electronics 138c and / or the sense electronics 138d). For example, a first set of magnetic attachments 224, 244 is for power (positive) and a second set of magnetic attachments 224, 244 is for ground. The magnetic attachments 224, 244 may also transmit audio signals in conjunction with conducting power via conduction from the head mounted display unit 210 to the removable speaker unit 232.

[0037] Alternatively, power may be conducted from the head mounted display unit 210 to the detachable speaker unit 232 using electrical contacts that are separate from the magnetic attachments 224, 244, which may also transmit audio signals. In other variations, audio signals may be transmitted independent of power, such as with separate contacts and / or optical connections. In embodiments of the detachable speaker unit 232 having sensing electronics 138d, sensor signals may be transmitted from the detachable speaker unit 232 to the head mounted display unit 210 using the magnetic attachments 224, 244, separate contacts, optical connections, or wirelessly. Power may also be provided inductively from the head mounted display unit 210.

[0038] 3A-3E, the display system 300 includes an audio headset 330 having a removable speaker unit 332 that can be releasably coupled to a head mounted display unit 310. The head mounted display unit 310 is configured as the head mounted display unit 110 and further includes a magnetic attachment 324 having a female configuration, such as a socket. The removable speaker unit 332 is configured as the removable speaker unit 132 and further includes a magnetic attachment 344 having a male configuration, such as a ball, that is received by the magnetic attachment 324 of the head mounted display unit 310. The magnetic attachments 324, 344 of the head mounted display unit 310 and the removable speaker unit 332 may be referred to as an HMD or female magnetic attachment and a speaker or male magnetic attachment, respectively.

[0039] 3A-3E, the head mounted display unit 310 includes one HMD magnetic attachment 324 positioned above a user's ear E. As shown, the HMD magnetic attachment 324 is coupled to the support 112 of the head mounted display unit 310 above the user's ear E. The speaker magnetic attachment 344 is coupled to the housing 134 of the removable speaker unit 332, for example, formed at the end of an elongated protrusion (e.g., shaft) extending upwardly from the housing 134.

[0040] The magnetic attachments 324, 344 have complementary profiles that allow for reception of the speaker magnetic attachment 344 by the HMD magnetic attachment 324, such as a spherical shape. The magnetic attachments 324, 344 may also be configured to allow movement of the detachable speaker unit 332 relative to the head mounted display unit 310 while still being supported thereby. For example, the detachable speaker unit 332 may be allowed to displace inward and outward, forward and backward, and rotate, as shown by the arrows in Figures 3A and 3B.

[0041] The head mounted display unit 310 and the detachable speaker unit 332 are additionally configured relative to the head mounted display unit 310 to provide power to the detachable speaker unit 332. As shown, the magnetic attachments 324, 344 conduct electricity from the head mounted display unit 310 to the detachable speaker unit 332 to power the amplifier 138a and / or other electronics 138. For example, each of the magnetic attachments 324, 344 is subdivided to provide power and ground contacts that maintain physical contact for conducting electricity as the detachable speaker unit 332 moves relative to the head mounted display unit 310.

[0042] For example, as shown in FIG. 3C, the HMD magnetic attachment 324 may include a first power contact 324a (e.g., positive) and a second power contact 324b (e.g., ground) that are spaced apart along the spherical surface of the HMD magnetic attachment 324 and electrically insulated from one another. As shown in FIG. 3D, the speaker magnetic attachment 344 includes a first power contact 344a (e.g., positive) and a second power contact 344b (e.g., ground) that are electrically insulated from one another and have a larger conductive surface area than their corresponding first power contact 324a and second power contact 324b of the HMD magnetic attachment 324. As a result, as shown in FIG. 3E, the detachable speaker unit 332 can move inward / outward, forward / backward, and / or rotate with a limited range of motion while still maintaining conductive contact between the power contacts 344a, 344b and 324a, 324b to conduct power therebetween. The magnetic attachments 324, 344 may also transmit audio signals in conjunction with conducting power from the head mounted display unit 310 to a detachable speaker unit 332. The power contacts 324a, 324b of the HMD magnetic attachment 324 may be, for example, permanent magnets or may be made of or otherwise include a ferromagnetic material, while the power contacts 344a, 344b of the speaker magnetic attachment 344 may be made of a ferromagnetic material or may be permanent magnets of opposite orientation to those of the HMD magnetic attachment 324.

[0043] Although the detachable speaker units 232, 332 have been described as being movable relative to the head mounted display units 210, 310, variations of the magnetic attachments 224, 244 and magnetic attachments 324, 344 may instead be configured to prevent movement therebetween.

[0044] Additionally, although the detachable speaker units 232, 332 have been described as being magnetically releasably coupleable to the head mounted display units 210, 310, they may instead be coupled to one another with other releasable mechanisms, such as buckles, clamps, or fasteners, and in such cases, power and signals may be conducted through such releasable mechanisms or through separate power and data connections.

[0045] 4A-4C, the display system 400 includes a head mounted display unit 410 and an audio headset 430 having removable speaker units 432, such as a right removable speaker unit 432r and a left removable speaker unit 432l, that can be removably coupled to one another. Each of the removable speaker units 432 is configured as one of the removable speaker units 132, 232, 332 and further includes a coupling mechanism 449. In one embodiment, the coupling mechanism 449 couples the removable speaker units 432 to one another using magnetic forces. For example, the coupling mechanism 449 of the right removable speaker unit 432r may be or include a permanent magnet or a ferromagnetic material, while the coupling mechanism 449 of the left removable speaker unit 432l is or includes a permanent magnet (e.g., in an opposing orientation) or a ferromagnetic material and is configured to be magnetically attracted thereto.

[0046] When the coupling mechanism 449 is configured to removably couple the removable speaker units 432 to one another with magnetic forces, it may be hidden from view (eg, housed within the housing 134 or pad 246).

[0047] The coupling mechanism 449 may be a mechanical mechanism (eg, a buckle, clamp, or fastener) that removably couples the removable speaker units 432 to one another instead of or in addition to using magnetic forces.

[0048] 4C, the coupling mechanism 449 may also function to couple the removable speaker unit 432 to the head mounted display unit 410. For example, the head mounted display unit 410 may be configured as the head mounted display unit 110 and may include a coupling mechanism 429 corresponding to the coupling mechanism 449. For example, as shown in FIG. 4C, the coupling mechanism 429 on the right side of the head mounted display unit 410 may include one or more of the coupling mechanisms 429 in a similar arrangement (e.g., location, magnetic pole orientation if using magnetic forces, or common male / female configuration if using mechanical engagement) to the coupling mechanism 449 of the left removable speaker unit 432l. As a result, the right side of the head mounted display unit 410 may be coupled to the right removable speaker unit 432r in a similar manner as the left removable speaker unit 432l is coupled to the right removable speaker unit 432r.

[0049] 5-7B, for any of the display systems 100, 200, 300, 400, the head mounted display unit 110, 210, 310, 410 and the audio headset 130, 230, 330, 430, or both, output sound according to the state of the audio headset 130, 230, 330, 430. For simplicity of the following description, only the display system 100 will be referenced, but it should be understood that the other display systems 200, 300, 400 may be configured as described with respect to FIGS. 5-7B. As described in more detail below, the state of the audio headset 130 according to which sound is output may be one or more of: whether the detachable speaker units 132 are coupled together (i.e., forming an audio unit as described above), or the position or relative movement of the audio headset 130 or the detachable speaker unit 132 with respect to the head mounted display unit 110.

[0050] 5, in one embodiment, the HMD speaker 118 of the head mounted display unit 110 outputs sound when the detachable speaker units 132 are separated from one another, e.g., after the detachable speaker units 132 are separated from one another and for a duration thereafter. In such a case, the sound output from the HMD speaker 118 of the head mounted display unit 110 can represent activation of the audio headset 130 or its detachable speaker unit 132 to provide an audio cue or feedback indicative of the sound output capabilities of the audio headset 130 (e.g., its detachable speaker unit 132).

[0051] 5 illustrates a method 500 for outputting sound from the HMD speakers 118 following separation of the detachable speaker units 132 from one another. The method may be implemented, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker units 132.

[0052] In a first operation 510, separation of the removable speaker units 132 from one another is determined. Separation of the removable speaker units 132 may occur, for example, by a user pulling the removable speaker units 132 apart. Separation of the removable speaker units 132 can be determined by the controller 150 using, for example, sensors in the head mounted display unit 110, such as by detecting a change in relative movement or proximity between the removable speaker units 132, or by processing video received from the camera 116 or sensing electronics 138d of the removable speaker units 132.

[0053] In a second operation 520, sound is output from the HMD speaker 118 of the head mounted display unit 110 based on the determined separation of the removable speaker unit 132. For example, the controller 150 may cause audio signals and / or power to be sent to the HMD speaker 118 to output sound therefrom. The sound output from the HMD speaker 118 includes auditory content. Such auditory content may include audio indicators associated with the removable speaker unit 132.

[0054] In a third operation 530, sound is output from the removable speaker unit 132 of the audio headset 130. For example, the controller 150 may cause an audio signal and / or power to be sent to the removable speaker unit 132. The sound output from the removable speaker unit 132 may include auditory content, such as associated with a virtual or mixed reality experience.

[0055] At operations 520, 530, sound may be output from the removable speaker unit 132 according to the methods described below with reference to Figures 6A-6B and 7A-7B.

[0056] 6A-6B and 7A-7B, audio signal processing is performed (e.g., by controller 150) such that HMD speaker 118 of head mounted display unit 110 and detachable speaker unit 132 output sound according to the position of audio headset 130 and / or its detachable speaker unit 132. As described in more detail below, such audio signal processing or other audio control may include adjusting volume (see FIGS. 6A-7B and related discussion), equalization (see FIGS. 8A-8B and related discussion), and / or dynamic range (see FIGS. 9A-9B).

[0057] 6A-6B, the removable speaker unit 132 may be configured not to output sound until it is coupled to the head mounted display unit 110. For example, the removable speaker unit 132 may not have a power source independent of the head mounted display unit 110 (e.g., not have an internal battery, not have a wired connection to a power source independent of the head mounted display unit 110, or be otherwise powered only when it is removably coupled to the head mounted display unit 110). In other embodiments, the removable speaker unit 132 may receive audio signals associated with the head mounted display unit 110 (e.g., virtual audio content) only when it is physically coupled thereto, such as via a conductive or optical data connection. In still further embodiments, the removable speaker unit 132 is otherwise configured (e.g., by software programming) not to output sound until it is removably coupled to the head mounted display unit 110.

[0058] In such a scenario, the HMD speakers 118 of the head mounted display unit 110 can output sound to simulate or otherwise represent the sound being emitted by the detachable speaker unit 132 while the detachable speaker unit 132 is not coupled to the head mounted display unit. Upon connecting the detachable speaker unit 132 to the head mounted display unit 110, the sound output from the detachable speaker unit 132 can be increased to become the primary sound source (e.g., turned on), while the sound output from the HMD speakers 118 can simultaneously or thereafter be substantially decreased (e.g., to emit no sound).

[0059] 6A shows a plot of the volume of sound output from the HMD speaker 118 of the head mounted display unit 110 and the detachable speaker unit 132 of the audio headset 130 as a function of the distance of the detachable speaker unit 132 to a predetermined docking position where the detachable speaker unit 132 can be docked to the audio headset 130. Zero distance (i.e., at the docking position) is shown on the right side of the plot so that moving rightward along the plot reflects moving the detachable speaker unit 132 towards the docking position. The plot also shows a docking state indicated by C, where the detachable speaker unit 132 is docked to the detachable speaker unit 132 at the docking position. It should be understood that although docking state C is shown after zero distance, this should be understood as representing a state change in the docking state of the detachable speaker unit 132 and does not necessarily represent a further change in distance between the detachable speaker unit 132 and the docking position. As used herein, the term "volume" generally refers to the level of sound output by an audio output device, and may also be referred to as "sound output level."

[0060] As shown, the volume 118v of the HMD speaker 118 increases as the detachable speaker unit 132 moves closer to the docked position, reaching a peak volume when the detachable speaker unit 132 is at or near the docked position. The volume 132v of the detachable speaker unit 132 remains at zero until the detachable speaker unit 132 reaches the docked position and / or is docked to the head mounted display unit 110.

[0061] In the docked position, or when in docked state C, the volume 132v of the detachable speaker unit 132 is increased, for example, to become the primary sound source for the user. Sound output from the speaker unit 132 is maintained while in docked state C.

[0062] Consistent with the increase in volume 132v of the detachable speaker unit 132, the volume 118v of the HMD speaker 118 decreases (e.g., when the detachable speaker unit 132 is moved to a docked position or docked to the head mounted display unit 110). For example, the volume 118v of the HMD speaker 118 can be zeroed and / or maintained at zero when the detachable speaker unit 132 is in the docked state C.

[0063] 6B illustrates a method 600 for outputting sound from HMD speakers 118 according to the position of audio headset 130 and / or detachable speaker unit 132, respectively, relative to head mounted display unit 110 (e.g., relative to a docking position on head mounted display unit 110). Method 600 may be implemented, for example, using controller 150 and various sensors in head mounted display unit 110 and detachable speaker unit 132. Although described with respect to one of detachable speaker units 132, it should be understood that method 600 is applicable to each of detachable speaker unit 132 and audio headset 130.

[0064] In a first operation 610, a change in the position of the removable speaker unit 132 relative to the head mounted display unit 110 (e.g., a docking position thereon) is determined. The position of the removable speaker unit 132 may change relative to the head mounted display unit 110, for example, when a user moves relative to the removable speaker unit 132 (e.g., rotates it toward, away from, the head) or when a user moves the removable speaker unit 132 (e.g., moves the removable speaker unit 132 to a docking position). A change in the position or relative movement of the removable speaker unit 132 may be determined by the controller 150 using, for example, a sensor of the head mounted display unit 110, such as by processing video received from the camera 116 (e.g., if the removable speaker unit 132 is powered independently of being docked to the head mounted display unit 110) or the sensing electronics 138d of the removable speaker unit 132, such as by detecting a movement toward the head mounted display unit and / or a change in proximity therebetween.

[0065] In a second operation 620, the volume 118v of the HMD speaker 118 is altered based on a change in the position of the detachable speaker unit 132 relative to the head mounted display unit 110. For example, as shown in FIG. 6A, the volume 118v increases or decreases as the detachable speaker unit 132 moves toward or away from the head mounted display unit 110 (e.g., a docking position thereon), respectively. The controller 150 can cause audio signals and / or power to be sent to the HMD speaker 118, causing sound to be output therefrom at the volume 118v. The sound output from the HMD speaker 118 can include auditory content, such as that associated with a virtual or mixed reality experience.

[0066] In a third operation 630, based on determining that the removable speaker unit 132 is at zero distance relative to the head mounted display unit 110 or upon coupling the removable speaker unit 132 to the head mounted display unit 110 in a predetermined position, the volume 132v of the removable speaker unit 132 is substantially increased. For example, the controller 150 may determine that the removable speaker unit 132 is at zero distance using sensors of the head mounted display unit 110 and / or the removable speaker unit 132 and / or by determining that the removable speaker unit 132 is coupled to the head mounted display unit 110, as described above with respect to the second operation 620. The controller 150 may cause an audio signal and / or power to be transmitted to the removable speaker unit 132 to output sound therefrom at a substantially increased volume 132v. For example, the volume 132v of the removable speaker unit 132 may be increased from zero to provide a primary or sole sound output to the user.

[0067] In a fourth operation 640, based on the determination of a zero distance coupling of the detachable speaker unit 132 or in accordance with the volume 132v of the detachable speaker unit 132 being substantially increased, the volume 118v of the HMD speaker 118 is substantially decreased (e.g., to zero). For example, the controller 150 may cause an audio signal and / or power to be transmitted to the HMD speaker 118 to decrease the volume 118v, or may not transmit to the HMD speaker 118 to decrease the volume 118v to zero. The fourth operation 640 may occur coincident with the third operation 630, such as simultaneously with or shortly thereafter.

[0068] In a fifth operation 650, while the detachable speaker unit 132 is coupled to the head mounted display unit 110 (e.g., in the coupled state shown in FIG. 6A ), the detachable speaker unit 132 continues to provide sound output to the user, while the HMD speaker 118 provides secondary sounds or no sound at all to the user.

[0069] In a sixth operation 660, acoustic transparency of the removable speaker unit 132 may be activated. Acoustic transparency is implemented by the ambient sound electronics 138e (e.g., microphone 138e′ and signal processor 138e”) to generate processed ambient sound as described above. Acoustic transparency may be activated automatically upon coupling the removable speaker unit 132 to the head mounted display unit 110, for example, when the display system 100 is operating in a mixed reality mode or by default, or may be activated by user selection. Acoustic transparency may provide a mixed reality experience by transmitting processed ambient sound to the user despite the removable speaker unit 132 physically preventing the ambient sound from flowing directly to the user's ears E.

[0070] In a seventh operation 670, noise cancellation of the removable speaker unit 132 may be activated. The noise cancellation is implemented by the ambient sound electronics 138e to generate a processed cancellation sound as described above. For example, when the display system 100 is operating in a virtual reality mode (e.g., when providing virtual visual content without visual pass-through) or by default (e.g., after a predetermined time after connecting the removable speaker unit 132 to the head mounted display unit 110 or after a predetermined time while operating acoustic transparency), noise cancellation may be activated automatically upon coupling the removable speaker unit 132 to the head mounted display unit 110 or may be activated by user selection. Noise cancellation can provide a virtual reality experience by further suppressing the user from hearing sounds from the real environment. It should be noted that noise cancellation may not be activated or may be activated without activating acoustic transparency (e.g., one, the other, or both of the sixth operation 660 and the seventh operation 670 may be performed).

[0071] In a variation of the method 600, determining the position of the removable speaker unit 132 (i.e., first operation 610) and modifying the volume 118v of the HMD speaker 118 based thereon (i.e., second operation 620), the first operation 610 and the second operation 620 may be omitted. In such a case, based on determining the zero position or combined, increasing the volume 132v of the removable speaker unit 132 (i.e., third operation 630) and decreasing the volume 118v of the HMD speaker 118 (i.e., fourth operation 640), the third operation 630 and the fourth operation 640 are still performed.

[0072] The method 600, and its previously described variations, may be used in conjunction with the previously described method 500, for example in the third operation 530.

[0073] 7A-7B, the removable speaker unit 132 may be configured to output sound before being coupled to the head mounted display unit 110. For example, the removable speaker unit 132 may be powered (e.g., by having an internal battery or a wired connection to a power source) independent of being coupled to the head mounted display unit. In such a scenario, the HMD speaker 118 of the head mounted display unit 110 and the removable speaker unit 132 may blend sound output to limit variations in volume experienced by a user, such as before the removable speaker unit 132 is coupled to the head mounted display unit 110.

[0074] The HMD speaker 118 of the head mounted display unit 110 and the detachable speaker unit 132 output sound individually according to the movement of the audio headset 130 (e.g., when the detachable speaker unit 132 is coupled to one another) and / or the movement of the detachable speaker unit 132. For example, when the detachable speaker unit 132 is at a relatively long distance from the head mounted display unit 110, sound is output only from the HMD speaker 118. When the detachable speaker unit 132 moves closer to the head mounted display unit 110, the detachable speaker unit 132 may increase the volume so that the user hears sounds emanating from both the HMD speaker 118 of the head mounted display unit 110 and the detachable speaker unit 132 simultaneously. As the detachable speaker unit 132 moves closer to the head mounted display unit 110, for example to couple the detachable speaker unit 132 thereto, the HMD speaker 118 and the detachable speaker unit 132 may cooperate to provide sound output such that volume fluctuations are limited by decreasing the volume from the HMD speaker 118 while simultaneously increasing and then decreasing the volume from the detachable speaker unit 132. This may help to limit the volume fluctuations perceived by the user.

[0075] 7A shows a plot of the volume of sound output from the HMD speaker 118 and the detachable speaker unit 132 as a function of the distance of the detachable speaker unit 132 to a predetermined docking position where the detachable speaker unit 132 can be docked to the audio headset 130. Zero distance (i.e., at the docking position) is shown to the right of the plot such that moving right along the plot reflects moving the detachable speaker unit 132 towards the docking position.

[0076] As shown, at a relatively far distance between the detachable speaker unit 132 and the docking location on the head mounted display unit 110, the HMD speaker 118 outputs sound at a volume 118v while the detachable speaker unit 132 does not output sound. At such a relatively far distance, the volume 118v of the HMD speaker 118 may be independent and / or constant from such distance. As the detachable speaker unit 132 moves closer to the docking location, the volume 118v of the HMD speaker 118 may decrease while the volume 132v of the detachable speaker unit 132 increases. At such a closer distance, the user can hear sound from the output from both the HMD speaker 118 and the detachable speaker unit 132. At close distances, such as when the detachable speaker unit moves to a zero distance coupled to the head mounted display unit 110 above the user's ear E, the volume 118v output by the HMD speaker 118 may decrease to zero while the volume 132v of the detachable speaker unit 132 may decrease to a final output level. The decrease in volume 132v of the detachable speaker unit 132 may take into account an increase in volume that the user might otherwise perceive due to the decreased distance and / or sound separation that may be perceived from the real environment due to, for example, the housing 134 or pads 246 of the detachable speaker unit 132 engaging the user's ear E.

[0077] 7B illustrates a method 700 for outputting sound from the HMD speakers 118 and the detachable speaker unit 132, individually according to the position of the audio headset 130 and / or the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., relative to a docking position on the head mounted display unit 110). The method 700 may be performed, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker unit 132. Although described with respect to only one of the detachable speaker units 132, it should be understood that the method 700 is applicable to each of the detachable speaker unit 132 and the audio headset 130.

[0078] In a first operation 710, the position of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., a predetermined docking position thereon) is determined, so that it can be performed in the manner described above for the first operation 610 of method 600.

[0079] In a second operation 720, based on a position of the removable speaker unit 132 relative to the head mounted display unit 110 that is relatively far away (e.g., greater than a threshold distance), the HMD speaker 118 outputs sound (i.e., volume 118v is greater than zero), while the removable speaker unit 132 does not output sound (i.e., volume 132v is zero). The controller 150 can cause audio signals and / or power to be sent to the HMD speaker 118 to output sound therefrom. The sound output from the HMD speaker 118 can include auditory content, such as that associated with a virtual or mixed reality experience.

[0080] In a third operation 730, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in an intermediate range (e.g., below the threshold distance and above the second threshold) and decreasing, the volume 118v of the HMD speaker 118 is decreased while the volume 132v of the detachable speaker unit 132 is simultaneously increased. The controller 150 can cause audio signals and / or power to be transmitted to the HMD speaker 118 and the detachable speaker unit 132.

[0081] In a fourth operation 740, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being close (e.g., below a second threshold distance to zero distance) and decreasing to zero distance, the volume 118v of the HMD speaker 118 is decreased to zero and the volume 132v of the detachable speaker unit 132 is decreased slightly to a non-zero value. The controller 150 causes audio signals and / or power to be transmitted to the HMD speaker 118 to output sound therefrom.

[0082] In a fifth operation 750, while the detachable speaker unit 132 is coupled to the head mounted display unit 110, the detachable speaker unit 132 continues to provide sound output to the user at a volume 132v greater than zero, while the HMD speaker 118 does not provide sound to the user.

[0083] In a sixth operation 760, the acoustic transparency of the removable speaker unit 132 may be activated, as described above for the sixth operation 670 of the method 600.

[0084] In a seventh operation 770, noise cancellation of the removable speaker unit 132 may be activated, as described above for the seventh operation 670 of the method 600.

[0085] It should be noted that aspects of method 600 may be incorporated into method 700, where the volume 118v of the HMD speaker 118 may vary to simulate movement of the detachable speaker unit 132 with sound emanating therefrom. For example, in the second act 720, the volume 118v may be increased to simulate the detachable speaker unit 132 moving closer to its docking position on the head mounted display unit 110, and in the third act 730, it may be increased or remain constant. Additionally, method 700 may be used in conjunction with the previously described method 500, for example in the third act 530.

[0086] 8A-8B, for a removable speaker unit 132 configured to output sound when detached from the head mounted display unit 110, equalization of the sound output by the removable speaker unit 132 may vary according to the distance from the removable speaker unit 132 to the head mounted display unit 110 (e.g., a docking location thereon). For example, as the removable speaker unit 132 moves closer to the head mounted display unit 110, the gain of lower frequency (e.g., bass range) sounds may be increased and the gain of higher frequency sounds may be decreased. Reducing the gain at low frequencies and increasing the gain at higher frequencies concentrates the power output for higher frequency sounds, which generally consume less power than lower frequency sounds for a given volume output. Because the power output of the detachable speaker unit 132 may be limited due to its portable / detachable nature (e.g., being battery powered) and / or primary use case (e.g., being held in close proximity to the user's ear E), equalization in this manner may allow the user to experience a louder sound while the detachable speaker unit 132 is at a relatively greater distance than would be the case if equalization were not performed.

[0087] 8A, the gain of low frequency sounds output by the removable speaker unit 132 is referred to as low frequency gain 832L, while the gain of higher frequency sounds output by the removable speaker unit 132 is referred to as high frequency gain 832H. The low frequency gain 832L and high frequency gain 832H are represented by a plot where the Y axis is gain and the X axis is the distance between the removable speaker unit 132 and the head mounted display unit 110 (e.g., the coupling position thereon), with zero distance on the right side (i.e., the removable speaker unit 132 moves closer to the coupling position moving rightward along the plot).

[0088] At a relatively long distance between the removable speaker unit 132 and the head mounted display unit 110, the low frequency gain 832L is below 1, e.g., starts at zero. As the distance decreases, such as when the user moves the removable speaker unit 132 toward the head mounted display unit 110 (e.g., to a coupling position above the user's ear E), the low frequency gain increases, e.g., ends at 1. Conversely, the high frequency gain 832H is above 1 at a relatively long distance and decreases as the distance decreases, e.g., ends at 1. As a result, at a relatively long distance, the low frequency gain 832L is below 1 and the high frequency gain 832H is above 1, and the difference between the low frequency gain 832L and the high frequency gain 832H decreases as the distance decreases, e.g., the difference ends at zero.

[0089] Alternatively or additionally, at relatively close distances, the gain for low frequency sounds may be increased above 1 at relatively close distances, such as when the removable speaker unit 132 is in the user's hand and being moved toward the head mounted display unit 110 coupled thereto. Such low frequency sounds may have a tendency to roll off (e.g., be attenuated) when the removable speaker unit 132 is held by the user, and increasing the gain for such low frequency sounds may provide a flatter sound, or the perception of a flatter sound, to the user. Thus, referring again to FIG. 8A , the alternative gain for low frequency sounds output by the removable speaker unit 132 is referred to as low frequency gain 832L′. At relatively close distances between the removable speaker unit 132 and the head mounted display unit 110, the low frequency gain 832L′ may be increased above 1 and may decrease, for example, gradually (as shown) or in a stepwise manner toward and / or up to 1 as the distance decreases. At relatively long distances, the low frequency gain 832L' may be the same as the low frequency gain 832L (e.g., starting below 1, such as at zero, and increasing as the distance decreases), or may be some other value (e.g., 1).

[0090] 8B illustrates a method 800 for outputting sound from detachable speaker unit 132 using equalization audio signal processing according to the position of audio headset 130 and / or detachable speaker unit 132 individually relative to head mounted display unit 110 (e.g., relative to a docking position on head mounted display unit 110). Method 800 may be performed, for example, using controller 150 and various sensors of head mounted display unit 110 and detachable speaker unit 132. Although described with respect to only one of detachable speaker units 132, it should be understood that method 800 is applicable to each of detachable speaker unit 132 and audio headset 130. Method 800 may be referred to as an equalization method or an equalization audio signal processing method and may be used alone and / or in conjunction with method 700.

[0091] In a first operation 810, the position of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., a predetermined docking position thereon) is determined, so that it can be performed in the manner described above for the first operation 610 of method 600.

[0092] In a second operation 820, based on a position of the removable speaker unit 132 being relatively far away (e.g., greater than a threshold distance) relative to the head mounted display unit 110, the low frequency gain 832L is less than 1 and / or the high frequency gain 832H is greater than 1, and sound is output from the removable speaker unit 132 according to the low frequency gain 832L and / or the high frequency gain 832H. The controller 150 may adjust the low frequency gain 832L and the high frequency gain 832H, for example, as shown in FIG. 8A, and cause audio signals and / or power to be sent to the removable speaker unit 132 to output sound therefrom according to such adjusted gains. The output sound may include auditory content, such as that associated with a virtual or mixed reality experience.

[0093] In a third operation 830, based on the position of the removable speaker unit 132 relative to the head mounted display unit 110 being in an intermediate range (e.g., below the threshold distance and above the second threshold) and decreasing, the low frequency gain 832L is increased and / or the high frequency gain 832H is decreased, and sound is output from the removable speaker unit 132 according to the (adjusted) low frequency gain 832L and / or high frequency gain 832H. The controller 150 may adjust the low frequency gain 832L and the high frequency gain 832H, for example, as shown in FIG. 8A, and cause audio signals and / or power to be sent to the removable speaker unit 132 to output sound therefrom according to such adjusted gains.

[0094] In a fourth operation 840, based on the position of the removable speaker unit 132 relative to the head mounted display unit 110 being in a relatively close range (e.g., below a second threshold) and decreasing, the low frequency gain may optionally be increased above 1, and then optionally further decreased towards 1 as the distance decreases to zero. The controller 150 may adjust the low frequency gain 832L′ and cause audio signals and / or power to be sent to the removable speaker unit 132 to output sound in accordance with such gain.

[0095] 9A-9B, for a removable speaker unit 132 configured to output sound when detached from the head mounted display unit 110, the dynamic range of the sound output by the removable speaker unit 132 can vary according to the distance of the removable speaker unit 132 relative to the head mounted display unit 110 (e.g., the coupling position thereon). Dynamic range generally refers to the volume range between loud and soft sounds, while dynamic range compression refers to audio signal processing that reduces the volume of louder sounds and / or amplifies quieter sounds. For example, as the removable speaker unit 132 moves closer to the head mounted display unit 110, dynamic range compression can be applied at a reduced level, thereby resulting in a lower dynamic range at relatively far distances and a higher dynamic range at lower distances. Applying dynamic range compression in this manner can enable a user to hear more sound and / or experience a consistently higher volume from the removable speaker unit 132, for example, at relatively far distances.

[0096] 9A, the dynamic range 932R of the sound output by the detachable speaker unit 132 is shown as a function of the distance between (e.g., the coupling position on) the detachable speaker unit 132 and the head mounted display unit 110. As the distance decreases, less dynamic compression is applied such that the dynamic range 932R increases.

[0097] 9B illustrates a method 900 for outputting sound from detachable speaker unit 132 using dynamic range compression audio signal processing according to the position of audio headset 130 and / or detachable speaker unit 132 individually relative to head mounted display unit 110 (e.g., relative to a docking position on head mounted display unit 110). Method 900 may be performed, for example, using controller 150 and various sensors of head mounted display unit 110 and detachable speaker unit 132. Although described with respect to only one of detachable speaker unit 132, it should be understood that method 900 is applicable to each of detachable speaker unit 132 and audio headset 130. Method 900 may be referred to as a dynamic range compression method or a dynamic range compression audio signal processing method and may be used alone and / or in conjunction with method 700 and / or method 800.

[0098] In a first operation 910, the position of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., a predetermined docking position thereon) is determined, so that it can be performed in the manner described above for the first operation 610 of method 600.

[0099] In a second operation 920, a relatively high level of dynamic range compression is applied to achieve a relatively low dynamic range based on a relatively distant position of the removable speaker unit 132 relative to the head mounted display unit 110 (e.g., greater than a threshold distance). The controller 150 may, for example, apply dynamic range compression to achieve the dynamic range 932R shown in FIG. 9A and cause audio signals and / or power to be transmitted to the removable speaker unit 132 to output sound therefrom in accordance with such dynamic range. The output sound may include auditory content, such as that associated with a virtual or mixed reality experience.

[0100] In a third operation 930, based on the position of the removable speaker unit 132 relative to the head mounted display unit 110 being in an intermediate range (e.g., below a threshold distance) and decreasing, the level of dynamic range compression applied is decreased to achieve a relatively high dynamic range. The controller 150 may, for example, apply dynamic range compression to achieve the dynamic range 932R shown in FIG. 9A and cause audio signals and / or power to be transmitted to the removable speaker unit 132 to output sound therefrom in accordance with such dynamic range.

[0101] 10A-10D, display system 100 may be configured to display visual indicators 1060 associated with audio headset 130. For example, before detachable speaker unit 132 is coupled to head mounted display unit 110, head mounted display unit 110 may provide visual content including visual indicators 1060, such as animations, icons, or other indicators, that may be in spatial proximity to audio headset 130. Visual indicators 1060 may suggest or otherwise indicate the sound output capabilities of audio headset 130 and / or provide instructions for its use. Although described with respect to display system 100 and its components (e.g., head mounted display unit 110 and audio headset 130), the concepts and methods described with respect to FIGS. 10A-10D are applicable to display systems 200, 300, 400.

[0102] 10A, the user can view the display screen 114 of the head mounted display unit 110 and can see the audio headset 130 in their field of view (e.g., via a video or optical pass-through as described above) as a real object in a mixed reality experience, or a virtual object in a virtual reality experience, where the audio headset 130 is depicted by a rendered graphic. For illustrative purposes, the audio headset 130 is placed on a table in the real environment. As shown in FIG. 10B, the display screen 114 can also display visual indicators 1060 that can be visually associated with the audio headset 130, for example, by being depicted in close proximity to it, such as surrounding and / or overlaying it.

[0103] The visual indicator 1060 suggests or otherwise communicates the audio capabilities and / or use of the audio headset 130. In one embodiment, the visual indicator is an audio-related icon, such as a sound wave (as shown), a depiction of traditional headphones, or other icon evocative of the audio capabilities or use of the audio headset 130. Alternatively or in addition, the visual indicator 1060 may be an animation of sound waves or the like emanating from the audio headset 130. The movement and / or intensity of the visual indicator 1060 (e.g., the sound wave animation) may be correlated to its associated sound, for example, by changing appearance in response to a sound being output. For example, the visual indicator 1060 may visually pulse according to the beat of the sound or music (e.g., faster or slower in accordance with a faster or slower beat of the sound, respectively) or according to the intensity of a sonic event (e.g., growing in size and / or pulsating faster in accordance with a real or virtual event expected to produce a loud and / or intense sound, such as an explosion).

[0104] The visual indicator 1060 may be positioned in spatial proximity (e.g., in close proximity, overlapping, and / or surrounding) to the audio headset 130 and / or the removable speaker unit 132, respectively. The position of the audio headset 130 may be determined in the manner described above, for example, by using sensors of the head mounted display unit 110 (e.g., the camera 116 for video recognition of the audio headset 130, or the motion sensor 120) and / or the sensing electronics 138d of the removable speaker unit 132 (e.g., a camera, a motion sensor, etc.).

[0105] The visual indicator 1060, or variations thereof, may, for example, display one or more diagrams illustrating how to use the audio headset 130 to suggest or otherwise communicate how to use the audio headset 130. With reference to FIG. 10C, for example, when the removable speaker unit 132 is not within the user's field of view, the visual indicator 1070 may guide the user in moving the removable speaker unit 132 to a docking position on the head mounted display unit 110. The visual indicator 1070 includes an illustration 1070a representing the removable speaker unit 132 and another illustration 1070b representing the docking position. The illustrations 1070a, 1070b may be displayed in a spatial relationship relative to one another that represents a physical spatial arrangement between the removable speaker unit 132 and the docking position on the head mounted display unit 110, thereby communicating to the user the movement required to dock the removable speaker unit 132. For example, as shown, illustration 1070a (i.e., representing removable speaker unit 132) may be positioned below and to the right of illustration 1070b (i.e., representing the docking location on head mounted display unit 110), thereby representing that the right removable speaker unit 132 is forward and lower than the docking location. The visual indication 1070 may alternatively or additionally include iconography (e.g., arrows), text (e.g., written text), and / or animation (e.g., depicting movement of illustration 1070a towards illustration 1070b) instructing the user where to move removable speaker unit 132 to dock with head mounted display unit 110.

[0106] 10D, a method 1000 for providing visual content on a display system, and in particular for providing a visual indicator associated with an audio output device. The display system may be, for example, one of the display systems 100, 200, 300, 400. The method 1000 may be performed, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the removable speaker unit 132. For simplicity, the method 1000 is described in relation to the display system 100, but it should be understood that it is applicable to the display systems 200, 300, 400. In addition, although generally described below with respect to the removable speaker unit 132, it should be understood that the method 1000 is applicable to each of the removable speaker unit 132, the audio headset 130, or another audio output device, such as an external speaker.

[0107] In a first operation 1010, a location of an audio output device (e.g., audio headset 130, removable speaker unit 132, or other audio output device) is determined. The location of the audio output device may be determined by controller 150 using, for example, sensors in head mounted display unit 110, such as by detecting movement toward head mounted display unit 110 and / or a change in proximity therebetween, such as by processing video received from camera 116 (e.g., if removable speaker unit 132 is powered independently of being coupled to head mounted display unit 110) or sensing electronics 138d of removable speaker unit 132.

[0108] In the second operation 1020, while the audio output device is within the field of view of the head mounted display unit, the visual indicator 1060 is displayed in spatial proximity to the audio output device. For example, the controller 150 may send a display signal to the head mounted display unit 110 to display the visual indicator 1060 on the display screen 114 according to the location determined in the first operation 1010. The visual indicator 1060 may visually communicate to the user the audio characteristics of the audio output device (e.g., showing an icon or animation resembling a sound wave or type of audio device, such as headphones) and / or instructions for using the audio output device (e.g., for coupling the removable speaker unit 132 to the head mounted display unit 110), as described above. The field of view of the head mounted display unit 110 is the portion of the real environment that the user can see at a given time.

[0109] In a third operation 1030, when the audio output device is outside the field of view of the display screen 114, a visual indicator 1070 is displayed to indicate the physical proximity of the audio output device to the head mounted display unit (e.g., to the coupling position of the removable speaker unit 132). For example, the controller 150 may send a display signal to the head mounted display unit 110 to display the visual indicator 1070 on the display screen 114 according to the position determined in the first operation 1010. As described above, the visual indicator 1070 may include an illustration 1070a representing the removable speaker unit 132 and an illustration 1070b representing the coupling position in spatial relationship thereto, which represents the physical spatial relationship (e.g., physical proximity) of the removable speaker unit 132 to the coupling position on the head mounted display unit 110. It should be noted that one, the other, or both of the second operation 1020 and the third operation 1030 may be performed.

[0110] In a fourth action 1040, which may occur simultaneously with the second action 1020 (i.e., when the audio output device is in the field of view) and / or the third action 1030 (i.e., when the audio output device is out of the field of view), virtual auditory content may be output by the HMD speakers 118, thereby spatially simulating the sound emanating from the audio output device (e.g., spatially balanced and / or varying in volume from left to right to represent the left and right orientation and / or distance of the audio output device, respectively). The virtual auditory content may correspond to, and for example be time-coincident with, the display of the visual indicator 1060 or the visual indicator 1070.

[0111] Although the method 1000 is described as providing both visual indicators 1060, 1070, the method 1000 may instead include operations for providing only one of the visual indicators 1060, 1070.

[0112] As mentioned above, method 1000 may be used with audio output devices other than audio headset 130 or detachable speaker units 132 associated with display system 100. For example, another audio output device may be an external speaker that is not physically coupleable to a head-mounted display unit.

[0113] Additionally, method 1000 may be used in conjunction with methods 500, 600, 700 described above. For example, as described with methods 500, 600, 700, 800, 900, the visual indicators 1060, 1070 may be displayed simultaneously with sound output from the HMD speaker 118 and / or the detachable speaker unit 132.

[0114] With reference to FIG. 11, the controller 150 may have the hardware configuration shown and described, or other suitable configurations. The controller 150 may include a processor 1151, a memory 1152, a storage device 1153, a communication interface 1154, and a bus 1155 connecting such components or devices for communication therebetween. The processor 1151 is operable to execute computer program instructions and perform operations described by such instructions. By way of example, the processor 1151 may be a central processing unit or other conventional processing device. The memory 1152 may be a volatile high-speed short-term information storage device (e.g., a random access memory module). The storage device 1153 may be a non-volatile information storage device (e.g., a hard drive). The communication interface 1154 is configured to transmit and / or receive signals (e.g., wired or wireless) to and / or from, for example, the head mounted display unit 110 and / or the audio headset 130.

[0115] The physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic systems. A physical environment, such as a physical park, includes physical objects such as physical trees, physical buildings, and physical people. People can sense and / or interact with the physical environment directly, such as through sight, touch, hearing, taste, and smell.

[0116] In contrast, a computer-generated reality (CGR) environment refers to a wholly or partially simulated environment that people sense and / or interact with via electronic systems. In a CGR, a subset of a person's body movements or representations thereof are tracked, and in response, one or more properties of one or more virtual objects simulated within the CGR environment are adjusted to behave with at least one law of physics. For example, a CGR system may detect a person's head rotation and in response adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds may change in a physical environment. In some circumstances (e.g., for accessibility reasons), adjustments to the property(s) of the virtual object(s) in the CGR environment may be made in response to a representation of body movements (e.g., voice commands).

[0117] A person may sense and / or interact with a CGR object using any one of these senses, including vision, hearing, touch, taste, and smell. For example, a person may sense and / or interact with an audio object that creates an audio environment with 3D or spatial breadth that provides the perception of a point sound source in 3D space. In another example, an audio object may enable audio transparency that selectively incorporates ambient sounds from the physical environment, with or without computer-generated audio. In some CGR environments, a person may sense and / or interact with only the audio object.

[0118] Examples of CGR include virtual reality and mixed reality.

[0119] A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input for one or more senses. A VR environment includes a number of virtual objects that a person can sense and / or interact with. For example, computer-generated images of trees, buildings, and avatars representing people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence in the computer-generated environment and / or through a simulation of a subset of the person's physical movements in the computer-generated environment.

[0120] In contrast to a VR environment, which is designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment that is designed to incorporate sensory input from, or representations of, the physical environment in addition to including computer-generated sensory input (e.g., virtual objects). On the virtual continuum, a mixed reality environment is anywhere between, but not including, a fully physical environment at one end and a virtual reality environment at the other end.

[0121] In some MR environments, computer-generated sensory inputs may respond to changes in sensory inputs from the physical environment. Also, some electronic systems for presenting MR environments can track the location and / or orientation of virtual objects relative to the physical environment to allow them to interact with real objects (i.e., physical items or representations thereof from the physical environment). For example, the system can account for movement so that a virtual tree appears stationary relative to the physical ground.

[0122] Examples of mixed reality include augmented reality and augmented virtuality.

[0123] An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are overlaid on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or semi-transparent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on the transparent or semi-transparent display, whereby a person uses the system to perceive the virtual objects overlaid on the physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system composites the images or videos with the virtual objects and presents the composite on the opaque display. A person uses the system to indirectly view the physical environment through the images or videos of the physical environment and perceive the virtual objects overlaid on the physical environment. As used herein, a video of a physical environment shown on an opaque display is referred to as a "pass-through video," meaning that the system uses one or more imaging sensors to capture images of the physical environment and uses those images in presenting the AR environment on the opaque display. Further alternatively, the system may include a projection system that projects virtual objects into the physical environment, for example as a hologram or onto a physical surface, so that a person using the system perceives the virtual objects as overlaid on the physical environment.

[0124] Augmented reality environments also refer to simulated environments in which a representation of a physical environment is transformed by computer-generated sensory information. For example, in providing pass-through video, the system may transform one or more sensor images to be imposed with a selected perspective (e.g., viewpoint) that differs from the perspective captured by the imaging sensor. As another example, the representation of the physical environment may be transformed by graphically altering (e.g., magnifying) a portion thereof, such that the altered portion is a version that represents the original captured image but is non-photorealistic. As a further example, the representation of the physical environment may be transformed by graphically removing or obscuring a portion thereof.

[0125] An augmented virtuality (AV) environment refers to a simulated environment in which a virtual or computer-generated environment incorporates one or more sensory inputs from a physical environment. The sensory inputs may be representations of one or more characteristics of the physical environment. For example, an AV park may have virtual trees and virtual buildings, while people with faces are realistically reproduced from images taken of physical people. As another example, virtual objects may adopt the shape or color of physical items imaged by one or more imaging sensors. As a further example, virtual objects may adopt shadows that match the position of the sun in the physical environment.

[0126] A wide variety of electronic systems exist that allow a person to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed over a person's eyes (e.g., similar to contact lenses), headphones / earphones, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablets, and desktop / laptop computers. A head-mounted system may have one or more speaker(s) and an integrated opaque display. Alternatively, the head-mounted system may be configured to accept an external opaque display (e.g., a smartphone). The head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment and / or one or more microphones for capturing audio of the physical environment. Rather than an opaque display, the head-mounted system may have a transparent or translucent display. A transparent or translucent display may have a medium through which light representing an image is directed to a person's eye. The display may utilize digital light projection, OLED, LED, uLED, liquid crystal on silicon, laser scanned light source, or any combination of these technologies. The medium may be a light guide, a holographic medium, an optical combiner, an optical reflector, or any combination of these. In an embodiment, the transparent or translucent display may be configured to be selectively opaque. A projection-based system may employ retinal projection technology that projects a graphical image onto a human retina. A projection system may also be configured to project virtual objects into the physical environment, for example, as a hologram or onto a physical surface.

[0127] As mentioned above, one aspect of the present technology is to collect and use data available from various sources to display visual and output audio content, which may include varying content based on the user's head or eye movements or state. The present disclosure contemplates that in some examples, this collected data may include personal information data that uniquely identifies a particular person or that can be used to contact or locate a particular person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, Twitter IDs, home addresses, data or records regarding the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), birth date, or any other identifying or personal information.

[0128] This disclosure recognizes that the use of such personal information data in the present technology may be used to the benefit of the user. For example, the personal information data may be used to output content, thus enabling an enhanced user experience for the user. Moreover, other uses of the personal information data that benefit the user are also contemplated by this disclosure. For example, health and fitness data may be used to provide insight into the overall wellness of the user, or may be used as proactive feedback to individuals using the technology to pursue wellness goals.

[0129] This disclosure contemplates that entities involved in the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will adhere to robust privacy policies and / or practices. Specifically, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or government requirements for keeping personal information data confidential and secure. Such policies should be easily accessible by users and should be updated as data collection and / or use changes. Personal information from users should be collected for the entity's lawful and legitimate use and should not be shared or sold except for those lawful uses. Furthermore, such collection / sharing should be conducted after informing and obtaining consent from the user. Furthermore, such entities should consider taking all necessary measures to protect and secure access to such personal information data and ensure that others having access to that personal information data adhere to their privacy policies and procedures. Furthermore, such entities may subject themselves to third-party assessments to attest to their adherence to widely accepted privacy policies and practices. Moreover, policies and practices should be adapted to the specific types of personal data collected and / or accessed, and should conform to applicable laws and standards, including jurisdiction-specific considerations. For example, in the United States, collection of or access to certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be addressed accordingly. Therefore, different privacy practices should be maintained with respect to different types of personal data in each country.

[0130] Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which a user selectively blocks use of or access to personal information data. That is, the present disclosure contemplates that hardware and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of observing the state or movement of the user's head or eyes, the present disclosure may be configured to allow the user to select "opt-in" or "opt-out" of participating in the collection of personal information data during registration for the service or at any time thereafter. In another example, the user may select not to provide user movement or state data or to limit the amount of time that user observation data is observed or maintained. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications regarding access or use of personal information. For example, the user may be notified upon download of an app that will access the user's personal information data, and then again immediately before the personal information data is accessed by the app.

[0131] Moreover, it is the intent of this disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintended or unauthorized access or use. Risk can be minimized by limiting data collection and deleting data when it is no longer needed. Furthermore, where applicable, de-identification of data can be used to protect user privacy in certain health-related applications. De-identification can be facilitated by removing certain identifiers (e.g., date of birth, etc.) where appropriate, controlling the amount or specificity of data stored (e.g., collecting location data at a city level rather than an address level), controlling how data is stored (e.g., aggregating data across users), and / or other methods.

[0132] Thus, while this disclosure broadly encompasses the use of personal information data to implement one or more of the various disclosed embodiments, this disclosure also contemplates that the various embodiments may be implemented without requiring access to such personal information data. That is, the various embodiments of the technology are not rendered inoperable by the absence of all or a portion of such personal information data. For example, content may be provided to a user by inferring preferences based on non-personal information data or a minimal amount of personal information, such as content requested by a device associated with the user or publicly available information.

Claims

1. a head mounted display unit for outputting visual content to a user; a detachable speaker unit detachably coupleable to the head mounted display unit for providing auditory content to the user; At least one of the visual content or the auditory content is altered according to a position of the removable speaker unit relative to the head mounted display unit. Display system.

2. The display system of claim 1 , wherein the head-mounted display unit includes built-in speakers, and the output of the auditory content is output from one or more of the removable speaker unit or the built-in speakers according to the position.

3. The display system of claim 2 , wherein a volume of the built-in speaker is varied according to the position of the removable speaker unit.

4. 4. The display system of claim 3, wherein the volume of the built-in speaker is positive when the removable speaker unit is not coupled to the head mounted display unit and is zero when the removable speaker unit is coupled to the head mounted display unit, and a separate volume of the removable speaker unit is positive when the removable speaker unit is coupled to the head mounted display unit.

5. The display system of claim 4 , wherein the other volume of the detachable speaker unit is positive when the detachable speaker unit is not coupled to the head mounted display unit.

6. 6. A display system as claimed in claim 2, wherein audio signal processing of the auditory content output by the detachable speaker unit changes as the detachable speaker unit is moved towards a docking position on the head mounted display unit.

7. 7. The display system of claim 6, wherein the audio signal processing includes increasing the volume as the detachable speaker unit is moved toward the docking position.

8. 8. A display system as claimed in claim 6, wherein the audio signal processing includes equalisation comprising increasing low frequency gain as the detachable speaker unit is moved towards the docking position.

9. 9. A display system as claimed in claim 6, wherein the audio signal processing includes increasing the dynamic range as the detachable speaker unit is moved towards the docking position.

10. 10. The display system of claim 1 , wherein the visual content includes a visual indicator in spatial proximity to the removable speaker unit when the removable speaker unit is within the field of view of the head mounted display unit.

11. The display system of claim 10 , wherein the visual indicator indicates to the user the sound output capabilities of the removable speaker unit.

12. 12. A display system as claimed in any one of claims 1 to 11, wherein the visual content includes a visual indicator of the physical proximity of the removable speaker unit to a coupling position at which the removable speaker unit can be removably coupled to the head mounted display unit to provide the auditory content to the user when the removable speaker unit is out of the field of view of the head mounted display unit.

13. 13. A display system as claimed in any one of claims 1 to 12, wherein the first and second detachable speaker units receive audio signals and power from the head mounted display unit when coupled to the head mounted display unit.

14. 14. The display system of claim 1, wherein the first and second detachable speaker units are powered and receive audio signals when detached from the head mounted display unit.

15. 15. A display system as claimed in any preceding claim, wherein the first and second detachable speaker units are detachably coupleable to the head mounted display unit using magnetic attachment.

Citation Information

Patent Citations

  • Head-mounted display device

    JP2007266870A

  • Voice input output device

    JP2015018384A

  • Strap System for Head-Mounted Displays

    US20160363771A1