Display system with audio output device

Abstract

A display system with an audio output device is disclosed. For example, a display system including a head-mounted display unit and a removable speaker unit is disclosed. The head-mounted display unit outputs visual content to the user and provides visual delivery of the real environment to the user. The detachable speaker unit can be detachably coupled to the head-mounted display unit for providing auditory content to a user. At least one of the visual content or the auditory content changes according to a position of the detachable speaker unit relative to the head-mounted display unit.

Description

[0001] This application is a divisional application of invention patent application 201880094477.7, filed on September 27, 2018, and entitled “Display system with audio output device”.

[0002] Cross-references to related patent applications

[0003] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 684,826, filed on June 14, 2018, the entire disclosure of which is incorporated herein by reference. Technical Field

[0004] The present disclosure relates to a display system, and in particular to an audio output device for the display system. Background Art

[0005] Virtual reality systems provide content to users that may include visual content and auditory content. The visual content may be provided by a head mounted display unit (HMD), while the auditory content is often provided by headphones that are physically separate from the head mounted display unit. Summary of the invention

[0006] Disclosed herein is a specific implementation of a display system. In one aspect, the display system includes a head-mounted display unit and a detachable speaker unit. The head-mounted display unit outputs visual content to a user and provides a visual representation of a real environment to the user. The detachable speaker unit can be detachably coupled to the head-mounted display unit for providing auditory content to the user. At least one of the visual content or the auditory content changes according to a position of the detachable speaker unit relative to the head-mounted display unit.

[0007] The head mounted display may include built-in speakers. Depending on the position, auditory content is output by one or more of the detachable speaker unit or the built-in speakers. Audio signal processing of the auditory content output by the detachable speaker unit may change as the detachable speaker unit moves toward the coupling position on the head mounted display unit. The audio signal processing includes changing one or more of the volume, balance, or dynamic range of the auditory content output by the detachable speaker unit. When the detachable speaker unit is in the field of view of the head mounted display unit, the visual content may include a visual indicator that is spatially close to the detachable speaker unit. The visual indicator may indicate to the user the sound output capability of the detachable speaker unit.

[0008] In another aspect, a display system includes a head mounted display unit and an audio headset. The head mounted display unit includes a first built-in speaker and a second built-in speaker. The audio headset includes a first detachable speaker unit and a second detachable speaker unit that can be detachably coupled to the head mounted display unit.

[0009] In another aspect, a method for providing content using a display system is provided, the display system having a head-mounted display unit, a controller, a sensor, and a movable audio output device that can move relative to the head-mounted display unit. The method includes determining, by the controller, a position of the movable audio output device using the sensor, and providing content using the head-mounted display unit based on the position of the movable audio output device. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1A A side view of the display system.

[0011] Figure 1B for Figure 1A Top view of the display system.

[0012] Figure 1C for Figure 1A Schematic diagram of the display system.

[0013] Figure 1D for Figure 1A Schematic diagram of a detachable speaker unit of a display system.

[0014] Figure 2A FIG. 1 is a partial side view of another display system that is a variation of the display system of FIG. 1 .

[0015] Figure 2B for Figure 2A Partial rear view of the display system.

[0016] Figure 3A FIG. 1 is a partial side view of another display system which is a variation of the display system of FIG. 1 .

[0017] Figure 3B for Figure 3A Partial rear view of the display system.

[0018] Figure 3C for Figure 3A A partial side view of a head-mounted display unit of a display system.

[0019] Figure 3D for Figure 3A A partial side view of a detachable speaker unit of a display system.

[0020] Figure 3E for Figure 3A A partial side view of a display system in which hidden features are shown in phantom lines.

[0021] Figure 4A A first view of the audio headset in a first state.

[0022] Figure 4B for Figure 4A A second view of the audio headset in a second state.

[0023] Figure 4C To include Figure 4A The audio headset is used as Figure 1A A side view of a display system of a variation of a display system.

[0024] Figure 5 is a flow chart of a method for outputting sound from the display system of FIG. 1 .

[0025] Fig. 6A is a graph of the sound output of the display system from FIG. 1 .

[0026] Figure 6B For use according to Fig. 6A A graph is provided and a flow chart of a method for providing sound output from the display system of FIG. 1 is provided.

[0027] Fig. 7A is a graph of the sound output of the display system from FIG. 1 .

[0028] Figure 7B For use according to Fig. 7A A graph is provided and a flow chart of a method for providing sound output from the display system of FIG. 1 is provided.

[0029] Fig. 8A is a graph of equalized gain versus distance for the sound output from the display system of FIG. 1 .

[0030] Figure 8B For use according to Fig. 8A The graph is a flow chart of a method of providing sound output from the display system of FIG1 .

[0031] Fig.9A is a graph of dynamic range versus distance for dynamic range compression of sound output from the display system of FIG. 1 .

[0032] Fig. 9B For use according to Fig.9A A graph is provided and a flow chart of a method for providing sound output from the display system of FIG. 1 is provided.

[0033] Fig. 10A A view of a head mounted display unit having a field of view including a real environment and an audio output device.

[0034] Fig. 10B To have Fig. 8A Another view of the head mounted display unit that includes a field of view and additionally displays a visual indicator.

[0035] Fig. 10C Another view of a head mounted display unit having a field of view that does not include an audio output device and that additionally displays another visual indicator.

[0036] Fig. 10D To provide Fig. 10B and Fig. 10C A flow chart of a method of providing a visual indicator.

[0037] Fig.11 for Figure 1A A schematic diagram of the hardware configuration of the display system's controller. DETAILED DESCRIPTION

[0038] Embodiments of a display system are disclosed herein, which include a head-mounted display unit and an audio output device, such as an audio headset. The audio output device can be detached from the head-mounted display unit in various ways described herein or can be moved relative to the head-mounted display unit in other ways. The auditory content provided from the audio output device and / or the built-in audio output device can be output in various ways described herein based on the position of the movable audio output device. The visual content can also be provided in various ways described herein based on the position of the movable audio output device.

[0039] Reference Figures 1A to 1C , the display system 100 includes a head mounted display unit 110 and an audio headset 130 that can be physically moved relative to the head mounted display unit 110 (such as being detachable therefrom (as shown) or articulated relative thereto). As discussed in further detail below, the display system 100 can form a mechanical connection and / or an electrical connection between the head mounted display unit 110 and the audio headset 130, sound can be provided to the user by the head mounted display unit 110 itself and the audio headset 130, and / or the head mounted display unit 110 can be used to provide prompts and / or feedback related to the use of the audio headset 130 to the user. The term "audio headset 130" refers to an audio output device or a set of audio output devices (e.g., a detachable speaker unit 132, as described below) that is to be held in close proximity to one or both ears of the user. In the case where there are multiple audio output devices (e.g., two), the audio output devices of the audio headset 130 can be separated from each other or can be separated from each other (e.g., without being attached to each other, such as with a headband extending between them). Audio earphones 130 may also be referred to as headphones or earbuds.

[0040] As described below, the display systems disclosed herein can be configured to provide a computer-generated reality (e.g., virtual reality or mixed reality). As used, the term "virtual" generally refers to "computer-generated" content or experience. For example, the content described herein can be referred to as virtual or computer-generated content, such as visual content (e.g., also referred to as virtual or computer-generated visual content or graphics) and auditory content (e.g., also referred to as virtual or computer-generated auditory content or audio).

[0041] The head mounted display unit 110 generally includes a support 112 and a display screen 114. The support is configured to be worn on the user's head H to support the display screen 114 in a suitable position relative to the user's eyes. As shown, the support is configured as a band around the user's head H. The display screen 114 (such as a screen of a smart phone or a dedicated screen) provides visual content to the user. In the case where the display screen 114 is a removable smart phone or other video output device, the head mounted display unit 110 can be considered to be able to couple to the display screen 114.

[0042] The display system 100 may be configured to provide a virtual reality experience by providing a visual transmission through which a user can view a real environment. As shown, the head-mounted display unit 110 utilizes video interleaving, in which case one or more cameras 116 of the head-mounted display unit 110 capture an image of the real environment displayed to the user by the display screen 114. Such a display system 100 may also be used to provide a virtual reality experience, in which case an image of the real environment is not displayed by the display screen 114 and the real environment is instead partially or completely blocked relative to the user's field of view. As an alternative to video interleaving, the head-mounted display unit 110 may utilize optical interleaving, in which case the user directly views the real environment. For example, the head-mounted display unit 110 may include a reflector that allows light from the real environment to pass through it to reach the user's eyes and reflects light from the display screen 114 to the user's eyes to provide visual content.

[0043] The head mounted display unit 110 may also include a speaker 118 that transmits or outputs auditory content to the user's ear. For example, the speaker 118 is coupled to the support 112 close to the user's ear E and directs the sound thereto. The speaker 118 is fixedly coupled to the head mounted display unit 110 (e.g., coupled to the housing of the support 112 or the display screen 114) so ​​that it cannot be easily removed therefrom. This is in contrast to the speaker unit 132 of the audio headset 130, which can be detachably coupled to the head mounted display unit 110 so that it can be easily removed therefrom. Therefore, the speaker 118 may be referred to as a fixed audio output device, a fixed speaker, a built-in speaker, or an HMD speaker, and the speaker unit 132 may be referred to as a detachable speaker or a detachable speaker unit. The audio headset 130 and the detachable speaker unit 132 may also be referred to as an audio output device or a removable audio output device.

[0044] The HMD speakers 118 also allow sounds from the real environment to reach the user's ears. For example, the HMD speakers 118 may be positioned above and / or in front of the user's ears so as not to prevent sounds from the real environment from reaching the user's ears. Thus, the head mounted display unit 110 may provide the user with a virtual reality experience in an auditory form by allowing the user to hear the real environment while transmitting auditory content to the user. Therefore, the HMD speakers 118 may also be referred to as additional auditory speakers.

[0045] As discussed in further detail below, the HMD speakers 118 are also configured to allow audio earphones 130 to provide auditory content to the user in conjunction with or excluding the HMD speakers 118. In some embodiments, the HMD speakers 118 may be omitted, in which case the auditory content is provided by the audio earphones 130.

[0046] The head mounted display unit may also include various sensors that detect conditions related to the user, the head mounted display unit 110, and / or the audio headset 130. For example, in addition to the camera 116, the head mounted display unit 110 may include a motion sensor 120 that measures conditions related to the position and / or orientation of the user's head H and / or the head mounted display unit 110, such as one or more of an accelerometer, a gyroscope, a magnetometer, an inertial measurement unit (IMU), a camera, etc. The camera 116 and / or the motion sensor 120 may also be configured to determine the position, orientation, and / or movement of the audio headset 130 relative thereto, for example using visual recognition of the audio headset 130, to detect, locate, and / or track a marker of a detachable speaker unit 132, and / or to detect, locate, and / or track a beacon signal emitted from the detachable speaker unit 132. The head mounted display unit 110 may also include an eye sensor 122, such as a camera, that is used to measure conditions of the user's eyes (e.g., focal length, pupil size, etc.).

[0047] The audio headset 130 includes a detachable speaker unit 132 for providing auditory content to the user by transmitting sound to the user's ear E. The detachable speaker unit 132 includes, for example, a detachable right speaker unit 132r and a detachable left speaker unit 132l to be physically associated with the user's right ear E and left ear E, respectively, for providing auditory content thereto.

[0048] Each of the detachable speaker units 132 generally includes a housing 134, one or more speakers 136, and various types of electronic devices 138. The housing 134 contains one or more speakers 136 and the electronic device 138 and is configured to be connected to the support 112 of the head mounted display unit 110 to be supported by it, receive audio signals from it, and / or receive electrical power from it. The speaker 136 outputs sound (i.e., the sound of the auditory content) to the user. The electronic device 138 is configured in conjunction with the speaker 136 to provide sound output, and can provide other functions.

[0049] As shown, the audio headset 130 may have an earmuff configuration, in which case the housing 134 is configured to surround the entire ear of the user. Alternatively, the audio headset 130 may have an on-ear configuration, in which case the housing 134 is configured to be placed on the ear of the user, or have an in-ear configuration, in which case a portion of the housing 134 is positioned in the ear E of the user (e.g., partially in the ear canal).

[0050] like Figure 1DAs shown schematically, the electronic device 138 of the speaker unit 132 may include an amplifier 138a, a power electronic device 138b, a signal electronic device 138c, a sensing electronic device 138d, and / or an ambient sound electronic device 138e. The amplifier 138a is configured to drive the speaker 136 for outputting sound. The power electronic device 138b is configured to power various other electronic devices of the detachable speaker unit 132, such as the amplifier 138a, the signal electronic device 138c, and / or the sensing electronic device 138d. The power electronic device 138b may, for example, provide wired power and / or locally stored power, such as using a battery. The signal electronic device 138c is configured to receive signals from the head-mounted display unit 110 or other components (such as the controller 150 discussed below) and / or send signals to the head-mounted display unit 110 or other components. The signal electronic device 138c may, for example, include a communication interface for sending and / or receiving signals, such as via a wireless, conductive or optical connection. Such signals may, for example, include audio signals received for outputting auditory content from the detachable speaker unit 132 and / or sensing signals for providing sensing information (e.g., position) of the detachable speaker unit 132. The signal electronics 138c may also provide outputs, for example by providing passive or active markers (e.g., visible or invisible light sources) or beacons (e.g., emitting another type of detectable and locatable output signal) to help determine the position and / or orientation of the detachable speaker unit 132 relative to the head mounted display unit 130. The sensing electronics 138d is configured to sense various parameters of the detachable speaker unit 132, such as the position, orientation, and / or movement (e.g., changes in position and orientation) of the detachable speaker unit 132 in real space or relative to another component of the display system 100 (e.g., head mounted display unit 110, its support 112, or a coupling location or feature thereon). The sensing electronics 138d may include one or more appropriate sensors for detecting such position, orientation, and / or motion, such as one or more accelerometers, one or more gyroscopes, an inertial measurement unit (IMU), one or more proximity sensors, and / or a camera (e.g., for video recognition).

[0051] The ambient sound electronics 138e is used to detect ambient sound (e.g., using a microphone 138e' from a real environment), process the detected ambient sound (e.g., using a signal processor 138e"), and cause the speaker 136 to output processed sound based thereon (e.g., using an amplifier 138a). The processed sound may, for example, provide acoustic transparency and / or active noise cancellation. With acoustic transparency, ambient sound from the real environment is typically reproduced into processed sound without significant change, which may be referred to as processed ambient sound and output by the speaker 136. Thus, for example, when the detachable speaker unit 132 is placed against the user's ear E and may otherwise obstruct (e.g., by isolating and / or blocking) the ambient sound of the real environment from directly reaching the user's ear E, the user may still hear the real environment. With active noise cancellation, the ambient sound is processed to provide a processed sound that substantially cancels or counteracts the ambient sound, which may be referred to as processed cancellation sound and output by the speaker 136 (e.g., by having the same amplitude as the ambient sound but opposite phase).

[0052] It should be noted that the speaker electronics 138 (e.g., amplifier 138a, power electronics 138b, signal electronics 138c, sensing electronics 138d, and / or ambient sound electronics 138e) or different electronics that perform similar functions may instead or additionally be incorporated into the head mounted display unit 110. For example, the speaker 136 may have a wired connection to the head mounted display unit 110 and / or the controller 150, while the amplification, power, sensing, and ambient sound functions are performed by the electronics of the head mounted display unit 110 and / or the controller 150.

[0053] In addition to the head mounted display unit 110 and the audio headset 130, the display system 100 also includes a controller 150. As shown, the controller 150 may be external to the head mounted display unit 110 and the audio headset 130 and communicate with them, such as using a wired or wireless connection. The controller 150 is generally configured to process various input information (e.g., signals), such as the position and / or orientation of the user, the head mounted display unit 110, and / or the audio headset 130, process such information, and provide output signals for controlling the head mounted display unit 110 (e.g., to output visual content and auditory content) and the audio headset 130 (e.g., to output auditory content). The hardware configuration of the controller 150 is discussed below with reference to FIG. 9. Although depicted as a single unit separate from and external to the head-mounted display unit 110 and the audio headset 130, the controller 150 may instead include a plurality of such controllers or be subdivided into separate units that act cooperatively as the controller 150, and the controller 150 may be incorporated into one or more of the head-mounted display unit 110 and the detachable speaker unit 132.

[0054] As described above, the head-mounted display unit 110 and the audio headset 130 can be configured cooperatively to provide a mechanical, electrical and / or data connection between the head-mounted display unit 110 and the audio headset 130, to cooperatively provide sound to the user, and / or to utilize the head-mounted display unit 110 to provide the user with prompts and / or feedback related to the use of the audio headset 130.

[0055] Reference FIG. 2A to FIG. 4B , each of the detachable speaker units 132 of the audio headset 130 is detachable from the head mounted display unit 110. When attached to the head mounted display unit 110, each of the detachable speaker units 132 is physically supported by it and can also communicate with it and / or receive power from it.

[0056] Reference FIG. 2A to FIG. 2B , the display system 200 includes an audio headset 230 having a detachable speaker unit 232 that can be releasably coupled to a head-mounted display unit 210 using magnetic force. The head-mounted display unit 210 is configured as the head-mounted display unit 110, while additionally including one or more magnetic attachments 224 that can be referred to as HMD magnetic attachments. The detachable speaker unit 232 is configured as the detachable speaker unit 132, while including one or more magnetic attachments 244 corresponding to the HMD magnetic attachments 224 of the head-mounted display unit 210. The magnetic attachments 224, 244 of the head-mounted display unit 210 and the detachable speaker unit 232 can be referred to as HMD magnetic attachments 224 and speaker magnetic attachments 244, respectively. The magnetic attachments 224, 244 are configured to attract each other using magnetic force, for example, one of the magnetic attachments 224, 244 includes a permanent magnet, and the other magnetic attachment includes a ferromagnetic material or another permanent magnet of opposite orientation.

[0057] exist FIG. 2A to FIG. 2B In the example shown, the head mounted display unit 210 includes two HMD magnetic attachments 224, one HMD magnetic attachment in front of the user's ear E, and the other HMD magnetic attachment behind the user's ear E. As shown, the two HMD magnetic attachments 224 may be coupled to the support 112 of the head mounted display unit 210, for example, coupled to flanges 212a (e.g., protrusions) extending downward to positions in front of and behind the user's ear E. The two HMD magnetic attachments 224 include permanent magnets that may be incorporated into the support 112 of the head mounted display unit 210 or may be coupled to the support 112 in other ways.

[0058] The detachable speaker unit 232 (such as its housing 134 or its pad 246) can protrude (e.g., extend inward) from the support 112 (e.g., the flange 212a inward) to be positioned closely to the user, such as engaging the head H around the ear E (e.g., earmuff configuration) or engaging the ear E of the user (e.g., on-ear configuration). Therefore, the detachable speaker unit 232 can be considered to have an inner portion (e.g., pad 246 and / or housing 134) extending laterally inward (e.g., toward the user) from an outer portion (e.g., formed by the speaker magnetic attachment 244) coupled to the head mounted display unit 110. The pad 246 can engage and / or externally connect the ear E of the user (e.g., in an on-ear configuration and / or an earmuff configuration).

[0059] Each of the detachable speaker units 232 includes two speaker magnetic attachments 244 that can be magnetically coupled to the two HMD magnetic attachments 224 of the head mounted display unit 210. For example, the two speaker magnetic attachments 244 may extend in front of and behind the user's ears E to be magnetically coupled to the HMD magnetic attachments 224. The speaker magnetic attachments 244 may, for example, be elongated and extend forward and backward from the housing 134 of the detachable speaker unit 232. As shown in the figure, the speaker magnetic attachment 244 may be configured as a column including a ferromagnetic material or a permanent magnet of opposite orientation for being attracted to the permanent magnet of the HMD magnetic attachment 224 of the head mounted display unit 210. Alternatively, the HMD magnetic attachment 224 may include a ferromagnetic material, and the speaker magnetic attachment 244 includes a permanent magnet to be attracted thereto.

[0060] The head mounted display unit 210 and the detachable speaker unit 232 may also be configured to guide the detachable speaker unit 232 to a predetermined position relative thereto (e.g., a coupling position of the head mounted display unit 110 above the ear E of the user). For example, each set of magnetic attachments 224, 244 (e.g., a front and rear pair of magnetic attachments 224, 244) provides a magnetic force to guide the speaker magnetic attachment 244 of the detachable speaker unit 232 to the HMD magnetic attachment 224 of the head mounted display unit 210 accordingly. In addition, the magnetic attachments 224, 244 that do not correspond to each other may also be used to repel each other, for example, by having permanent magnets with the same orientation. Therefore, the detachable speaker unit 232 is pushed away from a non-predetermined position. These attractive and / or repulsive forces can be used to help the user guide the detachable speaker unit 132 to its corresponding coupling position on the head mounted display unit 210, which is outside the user's field of view, by providing tactile feedback to the user.

[0061] Instead of or in addition to using magnetic force to guide the detachable speaker unit 232 to the appropriate position, the head mounted display unit 210 and the detachable speaker unit 232 may include mechanical cooperation guidance features. For example, the head mounted display unit 210 may include a detent in the support 112 that receives and guides the speaker magnetic attachment 244 of the detachable speaker unit 232 to a predetermined position.

[0062] The detachable speaker unit 232 may be configured to pivot around the speaker magnetic attachment 244, for example, to be biased toward the user's ear E or otherwise adapted to the user's ear E. In one example, the speaker magnetic attachment 244 is a cylindrical post extending forward and backward (e.g., forming a pivot axis 244a). The cylindrical post rolls along the surface of the HMD magnetic attachment 224 or the support 112 of the head-mounted display unit 210 to pivot relative thereto. For another example, the housing 134 pivots relative to the speaker magnetic attachment 244, while the speaker magnetic attachment does not move relative to the HMD magnetic attachment 224 of the head-mounted display unit 210 (e.g., does not roll), thereby still allowing the housing 134 and the speaker 136 therein to pivot relative to the head-mounted display unit 210.

[0063] The pivot axis 244a of the detachable speaker unit 232 is formed, for example, by the speaker magnetic attachment 244. As shown, the pivot axis 244a can be arranged in the center position of the detachable speaker unit 232 (e.g., horizontally and vertically), or can be vertically offset (e.g., toward the upper end of the detachable speaker unit 232) and / or horizontally offset (e.g., toward the inside or outside of the detachable speaker unit 232). In addition, the pivot axis 244a can be arranged outside the support 112 of the head-mounted display unit 210 as shown.

[0064] The detachable speaker unit 232 may also be configured to be biased toward the user's ear E about the pivot axis 244a, such as using a spring or a magnet. FIG. 2A to FIG. 2B In the example shown, the detachable speaker unit 232 can be biased toward the user's ear E using magnetic force. The head mounted display unit 210 may include a magnet 228 in one orientation positioned above the user's ear E, while the detachable speaker unit 232 includes another magnet 248 in the opposite orientation above the user's ear E, for example, this is achieved by forming or being incorporated into an elongated member extending upward from the housing 134. The magnets 228, 248 of the head mounted display unit 210 and the detachable speaker unit 232 repel each other to bias the detachable speaker unit 232 around the pivot axis 244a and toward the user's ear E.

[0065] The head mounted display unit 210 and the detachable speaker unit 232 are also configured for the head mounted display unit 210 to provide power to the detachable speaker unit 232. As shown, the magnetic attachments 224, 244 conduct electricity from the head mounted display unit 210 to the detachable speaker unit 232 to provide power to the amplifier 138a and other electronic devices (e.g., signal electronic devices 138c and / or sensing electronic devices 138d). For example, the first group of magnetic attachments 224, 244 is used for power (positive), and the second group of magnetic attachments 224, 244 is used for grounding. The magnetic attachments 224, 244 can also be combined with the conduction of electrical power to transmit audio signals from the head mounted display unit 210 to the detachable speaker unit 232 via conduction.

[0066] Alternatively, electrical power may be conducted from the head mounted display unit 210 to the detachable speaker unit 232 using electrical contacts independent of the magnetic attachments 224, 244, which may also transmit audio signals. In other variations, audio signals may be transmitted independently of electrical power, such as using separate contacts and / or optical connections. In an embodiment of a detachable speaker unit 232 having a sensing electronics 138d, the sensor signal may be transmitted from the detachable speaker unit 232 to the head mounted display unit 210 using magnetic attachments 224, 244, separate contacts, optical connections, or wirelessly. Electrical power may also be provided inductively from the head mounted display unit 210.

[0067] Reference FIG. 3A to FIG. 3E , the display system 300 includes an audio headset 330 having a detachable speaker unit 332 that can be releasably coupled to a head mounted display unit 310. The head mounted display unit 310 is configured as the head mounted display unit 110, and additionally includes a magnetic attachment 324 having a female configuration, such as a socket. The detachable speaker unit 332 is configured as the detachable speaker unit 132, while additionally including a magnetic attachment 344 having a male configuration, such as a ball, which is received by the magnetic attachment 324 of the head mounted display unit 310. The magnetic attachments 324, 344 of the head mounted display unit 310 and the detachable speaker unit 332 may be referred to as an HMD or female magnetic attachment, and a speaker or male magnetic attachment, respectively.

[0068] exist FIG. 3A to FIG. 3EIn the example shown, the head mounted display unit 310 includes an HMD magnetic attachment 324 positioned above the user's ear E. As shown, the HMD magnetic attachment 324 is coupled to the support 112 of the head mounted display unit 310 above the user's ear E. The speaker magnetic attachment 344 is coupled to the housing 134 of the detachable speaker unit 332, for example, formed at the end of an elongated protrusion (e.g., a shaft) extending upward from the housing 134.

[0069] The magnetic attachments 324, 344 have a complementary profile, such as a spherical shape, that allows the HMD magnetic attachment 324 to receive the speaker magnetic attachment 344. The magnetic attachments 324, 344 can also be configured to allow the detachable speaker unit 332 to move relative to the head mounted display unit 310 while still being supported by it. For example, the detachable speaker unit 332 can be allowed to shift inward and outward, forward and backward, and rotate, such as FIG. 3A to FIG. 3B As indicated by the arrow in .

[0070] The head mounted display unit 310 and the detachable speaker unit 332 are also configured for the head mounted display unit 310 to provide electrical power to the detachable speaker unit 332. As shown, the magnetic attachments 324, 344 conduct power from the head mounted display unit 310 to the detachable speaker unit 332 to provide power to the amplifier 138a and its other electronics 138. For example, each of the magnetic attachments 324, 344 is subdivided to provide power and ground contacts that maintain physical contact when the detachable speaker unit 332 moves relative to the head mounted display unit 310 for conducting power.

[0071] For example, Figure 3C As shown, the HMD magnetic attachment 324 may include a first power contact 324a (eg, positive) and a second power contact 324b (eg, ground) spaced apart and electrically isolated from each other along the spherical surface of the HMD magnetic attachment 324. Figure 3D As shown, the speaker magnetic attachment 344 includes a first power contact 344a (e.g., positive) and a second power contact 344b (e.g., ground) that are electrically isolated from each other and have a larger conductive surface area than the first power contact 324a and the second power contact 324b of the HMD magnetic attachment 324 corresponding thereto. Figure 3EAs shown, the detachable speaker unit 332 can be moved inward / outward, forward / backward and / or rotated within a limited range of motion while still maintaining conductive contact between the power contacts 344a, 344b and 324a, 324b to conduct electrical power between them. The magnetic attachments 324, 344 can also transmit audio signals from the head mounted display unit 310 to the detachable speaker unit 332 in conjunction with conducting electrical power. The power contacts 324a, 324b of the HMD magnetic attachment 324 can, for example, be permanent magnets or made of or otherwise include ferromagnetic materials, while the power contacts 344a, 344b of the speaker magnetic attachment 344 can be made of ferromagnetic materials or be permanent magnets oriented opposite to those of the HMD magnetic attachment 324.

[0072] While the detachable speaker units 232, 332 are discussed as being movable relative to the head mounted display units 210, 310, variations of the magnetic attachments 224, 244 and 324, 344 may instead be configured to prevent movement therebetween.

[0073] Additionally, while the detachable speaker units 232, 332 are discussed as being releasably coupled to the head mounted display unit 210, 310 using magnetic forces, they may instead be coupled to each other using other releasable mechanisms such as buckles, clamps, or fasteners. In such cases, power and signals may be conducted through such releasable mechanisms or separate power and data connections.

[0074] Reference FIG. 4A to FIG. 4C , the display system 400 includes an audio headset 430 having detachable speaker units 432 (such as a detachable right speaker unit 432r and a detachable left speaker unit 432l) that can be detachably coupled to the head-mounted display unit 410 and coupled to each other. Each of the detachable speaker units 432 is configured as one of the detachable speaker units 132, 232, 332, and further includes a coupling feature 449. In one example, the coupling feature 449 uses magnetic force to couple the detachable speaker units 432 to each other. For example, the coupling feature 449 of the detachable right speaker unit 432r may be or may include a permanent magnet or ferromagnetic material, while the coupling feature 449 of the detachable left speaker unit 432l is configured to be magnetically attracted thereto, such as being or including a (e.g., relatively oriented) permanent magnet or ferromagnetic material.

[0075] The coupling features 449 can be hidden from view (eg, housed within the housing 134 or cushion 246) when configured to detachably couple the detachable speaker units 432 to each other using magnetic forces.

[0076] The coupling features 449 may instead or in addition to using magnetic forces be mechanical mechanisms (eg, clasps, clips, or fasteners) that removably couple the detachable speaker units 432 to one another.

[0077] See also Figure 4C , coupling feature 449 can also be used to couple detachable speaker unit 432 to head mounted display unit 410. For example, head mounted display unit 410 can be configured as head mounted display unit 110 and include coupling feature 429 corresponding to coupling feature 449. For example, Figure 4C As shown, the coupling features 429 on the right side of the head mounted display unit 410 can include one or more of the coupling features 429 in a similar arrangement (e.g., position, magnetic pole orientation (if magnetic force is used), or common male / female configuration (if mechanical engagement is used) as the coupling features 449 of the detachable left speaker unit 432l. Thus, the right side of the head mounted display unit 410 can be coupled to the detachable right speaker unit 432r in a manner similar to the coupling of the detachable left speaker unit 432l to the detachable right speaker unit 432r.

[0078] refer to Figures 5 to 7B For any one of the display systems 100, 200, 300, 400, the head mounted display unit 110, 210, 310, 410 and the audio earphone 130, 230, 330, 430 or both thereof output sound according to the status of the audio earphone 130, 230, 330, 430. For simplicity of the following discussion, only the display system 100 will be referenced, but it should be understood that the other display systems 200, 300, 400 may be referred to as well. Figures 5 to 7B As discussed in further detail below, the condition of the audio headset 130 according to which the sound is output may be one or more of the following: whether the detachable speaker units 132 are coupled to each other (i.e., forming the audio unit described above), or the position or relative movement of the audio headset 130 or the detachable speaker unit 132 relative to the head mounted display unit 110.

[0079] See also Figure 5 In one example, when the detachable speaker units 132 are decoupled from each other, for example, when the detachable speaker units 132 are decoupled from each other and for a period of time thereafter, the HMD speaker 118 of the head mounted display unit 110 outputs a sound. In such cases, the sound output from the HMD speaker 118 of the head mounted display unit 110 may represent activation of the audio headset 130 or its detachable speaker unit 132 to provide an audio prompt or feedback, thereby indicating the sound output capability of the audio headset 130 (e.g., its detachable speaker unit 132).

[0080] Figure 5 A method 500 for outputting sound from the HMD speaker 118 according to the decoupling of the detachable speaker units 132 from each other is shown. The method can be implemented, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker unit 132.

[0081] In a first operation 510, it is determined that the detachable speaker units 132 are decoupled from each other. The decoupling of the detachable speaker units 132 may occur, for example, due to a user pulling the detachable speaker units 132 apart. The decoupling of the detachable speaker units 132 may be determined, for example, by the controller 150 using sensors of the head mounted display unit 110 (such as by processing video received from the camera 116) or using sensing electronics 138d of the detachable speaker units 132 (such as by detecting relative movement therebetween or a change in proximity therebetween).

[0082] In a second operation 520, based on the determined decoupling of the detachable speaker unit 132, sound is output from the HMD speaker 118 of the head mounted display unit 110. For example, the controller 150 may cause an audio signal and / or power to be sent to the HMD speaker 118 to output sound therefrom. The sound output from the HMD speaker 118 includes auditory content. Such auditory content may include an audio indicator associated with the detachable speaker unit 132.

[0083] In a third operation 530, sound is output from the detachable speaker unit 132 of the audio headset 130. For example, the controller 150 may cause an audio signal and / or power to be sent to the detachable speaker unit 132. The sound output from the detachable speaker unit 132 may include auditory content, such as auditory content related to a virtual reality experience or a virtual reality experience.

[0084] In operations 520 and 530, the sound may be FIG. 6A to FIG. 6B as well as FIG. 7A to FIG. 7B The sound is output from the detachable speaker unit 132 in the manner described.

[0085] Reference FIG. 6A to FIG. 6B as well as FIG. 7A to FIG. 7B , (e.g., controller 150) performs audio signal processing so that the HMD speaker 118 and the detachable speaker unit 132 of the head mounted display unit 110 output sound according to the position of the audio earphone 130 and / or its detachable speaker unit 132. As discussed in further detail below, such audio signal processing or other audio control may include adjusting the volume (see 6A to 7B and related discussions), equilibrium (see FIG. 8A to FIG. 8B and related discussions), and / or dynamic range (see FIG. 9A to FIG. 9B ).

[0086] Reference FIG. 6A to FIG. 6B , the detachable speaker unit 132 may be configured not to output sound before being coupled to the head-mounted display unit 110. For example, the detachable speaker unit 132 may not have a power source independent of the head-mounted display unit 110 (e.g., not having an internal battery, not having a wired connection to a power source independent of the head-mounted display unit 110, or otherwise being powered only when detachably coupled to the head-mounted display unit 110). In other examples, the detachable speaker unit 132 may receive audio signals (e.g., virtual audio content) associated with the head-mounted display unit 110 only when physically coupled (such as via a conductive or optical data connection). In another example, the detachable speaker unit 132 is otherwise configured not to output sound before being detachably coupled to the head-mounted display unit 110 (e.g., due to software programming).

[0087] In such a case, when the detachable speaker unit 132 is not coupled to the head mounted display unit 132, the HMD speaker 118 of the head mounted display unit 110 may output a sound to simulate or otherwise indicate that the sound is emitted by the detachable speaker unit 132. When the detachable speaker unit 132 is connected to the head mounted display unit 110, the sound output from the detachable speaker unit 132 increases to become the main sound source (e.g., turned on), and the sound output from the HMD speaker 118 may be significantly reduced (e.g., until no sound is emitted) at the same time or subsequently.

[0088] Fig. 6A A graph showing the volume of the sound output from the HMD speaker 118 of the head mounted display unit 110 and the detachable speaker unit 132 of the audio headset 130 according to the distance of the detachable speaker unit 132 from the predetermined coupling position where the detachable speaker unit 132 can be coupled to the audio headset 130 is shown. Zero distance (i.e., at the coupling position) is shown on the right side of the graph, so that moving to the right along the graph reflects moving the detachable speaker unit 132 toward the coupling position. The graph also shows a coupled state indicated by C, in which the detachable speaker unit 132 is coupled to the detachable speaker unit 132 at the coupling position. The coupled state C is shown after the zero distance, but should be understood as indicating a state change of the coupling state of the detachable speaker unit 132, and does not necessarily indicate a further change in the distance between the detachable speaker unit 132 and the coupling position. As used herein, the term "volume" generally refers to the level of sound output by an audio output device, and may also be referred to as a "sound output level".

[0089] As shown, the volume 118v of the HMD speaker 118 increases as the detachable speaker unit 132 is moved closer to the coupling position, and reaches a peak volume when the detachable speaker unit 132 is at or near the coupling position. The volume 132v of the detachable speaker unit 132 remains at zero before the detachable speaker unit 132 reaches the coupling position and / or is coupled to the head mounted display unit 110.

[0090] The volume 132v of the detachable speaker unit 132 is increased, for example, to become a main sound source for the user at the coupled position or when in the coupled state C. The sound output from the speaker unit 132 is maintained when in the coupled state C.

[0091] The volume 118v of the HMD speaker 118 decreases in accordance with the increase in the volume 132v of the detachable speaker unit 132 (e.g., when the detachable speaker unit 132 is moved into the coupled position or when coupled to the head mounted display unit 110). For example, when the detachable speaker unit 132 is in the coupled state C, the volume 118v of the HMD speaker 118 may go to and / or remain at zero.

[0092] Figure 6B A method 600 for outputting sound from the HMD speaker 118 according to the position of the audio earphone 130 and / or the detachable speaker unit 132, respectively, relative to the head mounted display unit 110 (e.g., the coupling position on the head mounted display unit 110) is shown. The method 600 can be implemented, for example, using the controller 150, and various sensors of the head mounted display unit 110 and the detachable speaker unit 132. Although discussed in conjunction with one of the detachable speaker units 132, the method 600 should be understood to be applicable to each of the detachable speaker unit 132 and the audio earphone 130.

[0093] In a first operation 610, a change in the position of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., a coupling location thereon) is determined. For example, as a user moves relative to the detachable speaker unit 132 (e.g., toward, away from, turn the head) or as the user moves the detachable speaker unit 132 (e.g., moves the detachable speaker unit 132 to a coupling location), the position of the detachable speaker unit 132 may change relative to the head mounted display unit 110. The change in the position or relative movement of the detachable speaker unit 132 may be determined, for example, by the controller 150 using a sensor of the head mounted display unit 110 (e.g., by processing video received from the camera 116) or using the sensing electronics 138d of the detachable speaker unit 132 (e.g., if the detachable speaker unit 132 is not dependent on being coupled to the head mounted display unit 110 for power).

[0094] In a second operation 620, the volume 118v of the HMD speaker 118 is changed based on the change in the position of the detachable speaker unit 132 relative to the head mounted display unit 110. For example, Fig. 6A As shown, as the detachable speaker unit 132 moves toward or away from the head mounted display unit 110 (e.g., the coupling position thereon), the volume 118v is increased or decreased accordingly. The controller 150 may cause the audio signal and / or power to be sent to the HMD speaker 118 so that the sound is output therefrom at the volume 118v. The sound output from the HMD speaker 118 includes auditory content, such as auditory content related to a virtual reality experience or a virtual reality experience.

[0095] In a third operation 630, based on determining that the detachable speaker unit 132 is at zero distance relative to the head mounted display unit 110 or when the detachable speaker unit 132 is coupled to the head mounted display unit 110 at a predetermined position, the volume 132v of the detachable speaker unit 132 is significantly increased. For example, the controller 150 may use sensors of the head mounted display unit 110 and / or the detachable speaker unit 132 as described above with reference to the second operation 620, and / or by determining that the detachable speaker unit 132 is coupled to the head mounted display unit 110 to determine that the detachable speaker unit 132 is at zero distance. The controller 150 may cause an audio signal and / or power to be sent to the detachable speaker unit 132 so that sound is output therefrom at a significantly increased volume 132v. For example, the volume 132v of the detachable speaker unit 132 may be increased from zero to provide a primary or sole sound output to the user.

[0096] In a fourth operation 640, based on determining the zero distance, the coupling of the detachable speaker unit 132, or according to the volume 132v of the detachable speaker unit 132 having been significantly increased, the volume 118v of the HMD speaker 118 is significantly reduced (e.g., reduced to zero). For example, the controller 150 may cause an audio signal and / or power to be sent to the HMD speaker 118 to reduce the volume 118v, or not to be sent to the HMD speaker 118 to reduce the volume 118v to zero. The fourth operation 640 may occur in concert with the third operation 630, such as simultaneously therewith or shortly thereafter.

[0097] In a fifth operation 650, the detachable speaker unit 132 is coupled to the head mounted display unit 110 (e.g., in Fig. 6A , the detachable speaker unit 132 continues to provide sound output to the user, while the HMD speaker 118 provides auxiliary sound or does not provide any sound to the user.

[0098] In a sixth operation 660, acoustic transparency of the detachable speaker unit 132 may be activated. Acoustic transparency is implemented by the ambient sound electronics 138e (e.g., a microphone 138e' and a signal processor 138e") to produce processed ambient sound as described above. Acoustic transparency may be automatically activated, for example, when the display system 100 is operating in a virtual reality mode or by default when the detachable speaker unit 132 is coupled to the head mounted display unit 110, or may be activated by a user's selection. Acoustic transparency may provide a virtual reality experience by transmitting processed ambient sound to the user, even though the detachable speaker unit 132 physically blocks the ambient sound from traveling directly to the user's ear E.

[0099] In the seventh operation 670, noise cancellation of the detachable speaker unit 132 may be activated. Noise cancellation is implemented by the ambient sound electronics 138e to produce the processed cancellation sound as described above. Noise cancellation may be automatically activated when the detachable speaker unit 132 is coupled to the head-mounted display unit 110, for example, when the display system 100 is operating in a virtual reality mode (e.g., when providing virtual visual content without visual delivery) or by default (e.g., after a predetermined period of time has passed after the detachable speaker unit 132 is connected to the head-mounted display unit 110, or after a predetermined time of operating acoustic transparency), or may be activated by the user's selection. Noise cancellation can provide a virtual reality experience by further suppressing the user from hearing sounds from the real environment. It should be noted that noise cancellation may not be activated or may be activated without activating acoustic transparency (e.g., one, the other, or both of the sixth operation 660 and the seventh operation 670 may be performed).

[0100] In a variation of the method 600, the first operation 610 and the second operation 620 of determining the position of the detachable speaker unit 132 (i.e., the first operation 610) and changing the volume 118v of the HMD speaker 118 based on the position (i.e., the second operation 620) may be omitted. In such a case, the third operation 630 and the fourth operation 640 of increasing the volume 132v of the detachable speaker unit 132 (i.e., the third operation 630) and decreasing the volume 118v of the HMD speaker 118 (i.e., the fourth operation 640) based on determining the zero position or when coupled are still performed.

[0101] The method 600 and its above-described variations may be used in combination with the method 500 described above, for example, in the third operation 530 .

[0102] Reference FIG. 7A to FIG. 7B , the detachable speaker unit 132 may be configured to output sound before being coupled to the head mounted display unit 110. For example, the detachable speaker unit 132 may be powered independently of being coupled to the head mounted display unit (e.g., by having an internal battery or a wired connection to a power source). In such cases, such as before the detachable speaker unit 132 is coupled to the head mounted display unit 110, the HMD speaker 118 of the head mounted display unit 110 and the detachable speaker unit 132 may mix the sound output to limit the fluctuating volume experienced by the user.

[0103] The HMD speaker 118 and the detachable speaker unit 132 of the head mounted display unit 110 output sounds according to the movement of the audio earphone 130 (for example, when the detachable speaker units 132 are coupled to each other) and / or the respective movements of the detachable speaker units 132. For example, when the detachable speaker unit 132 is at a relatively large distance from the head mounted display unit 110, sounds are output only from the HMD speaker 118. As the detachable speaker unit 132 is moved closer to the head mounted display unit 110, the detachable speaker unit 132 may increase the volume so that the user hears sounds emitted from both the HMD speaker 118 and the detachable speaker unit 132 of the head mounted display unit 110. As the detachable speaker unit 132 is moved closer to the head mounted display unit 110, such as for coupling the detachable speaker unit 132 thereto, the HMD speaker 118 and the detachable speaker unit 132 may cooperatively provide sound output to limit volume fluctuations by reducing the volume from the HMD speaker 118 while increasing and then reducing the volume from the detachable speaker unit 132. This may help limit fluctuations in volume perceived by the user.

[0104] Fig. 7AA graph is shown of the volume of sound output from the HMD speaker 118 and the detachable speaker unit 132 as a function of the distance of the detachable speaker unit 132 from a predetermined coupling position where the detachable speaker unit 132 can be coupled to the audio headset 130. Zero distance (i.e., at the coupling position) is shown on the right side of the graph, such that moving to the right along the graph reflects moving the detachable speaker unit 132 toward the coupling position.

[0105] As shown, at a relatively far distance between the detachable speaker unit 132 and the coupling position on the head mounted display unit 110, the HMD speaker 118 outputs sound with a volume 118v, while the detachable speaker unit 132 does not output sound. At such a relatively far distance, the volume 118v of the HMD speaker 118 may be independent of such distance and / or constant. As the detachable speaker unit 132 is further moved closer to the coupling position, the volume 118v of the HMD speaker 118 may be reduced, while the volume 132v of the detachable speaker unit 132 increases. At such a closer distance, the user can hear the sound output from both the HMD speaker 118 and the detachable speaker unit 132. At a close distance, such as when moving to the zero distance where the detachable speaker unit 132 is coupled to the head mounted display unit 110 above the ear E of the user, the volume 118v output by the HMD speaker 118 is reduced to zero, while the volume 132v of the detachable speaker unit 132 may be reduced to the final output level. The reduction in volume 132v of the detachable speaker unit 132 may, for example, take into account an increase in volume that might otherwise be perceived by the user due to the reduced distance and / or isolation of sound that might be perceived from the real environment due to the housing 134 or pad 246 of the detachable speaker unit 132 engaging the user's ear E.

[0106] Figure 7B A method 700 for outputting sound from the HMD speaker 118 and the detachable speaker unit 132 according to the position of the audio earphone 130 and / or the detachable speaker unit 132, respectively, relative to the head mounted display unit 110 (e.g., the coupling position on the head mounted display unit 110) is shown. The method 700 can be implemented, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker unit 132. Although discussed in conjunction with only one of the detachable speaker units 132, the method 700 should be understood to be applicable to each of the detachable speaker unit 132 and the audio earphone 130.

[0107] In a first operation 710 , a position of the detachable speaker unit 132 relative to the head mounted display unit 110 (eg, a predetermined coupling location thereon) is determined, which may be performed in the manner described above with respect to the first operation 610 of method 600 .

[0108] In a second operation 720, based on the position of the detachable speaker unit 132 being relatively far (e.g., greater than a threshold distance) relative to the head mounted display unit 110, the HMD speaker 118 outputs sound (i.e., the volume 118v is greater than zero), while the detachable speaker unit 132 does not output sound (i.e., the volume 132v is zero). The controller 150 may cause an audio signal and / or power to be sent to the HMD speaker 118 to output sound therefrom. The sound output from the HMD speaker 118 includes auditory content, such as auditory content associated with a virtual or virtual reality experience.

[0109] In a third operation 730, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in the intermediate range (e.g., below the threshold distance and above the second threshold) and decreasing, the volume 118v of the HMD speaker 118 is decreased, while the volume 132v of the detachable speaker unit 132 is increased simultaneously. The controller 150 may cause audio signals and / or power to be sent to the HMD speaker 118 and the detachable speaker unit 132.

[0110] In a fourth operation 740, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in a very close distance (e.g., below the second threshold distance to the zero distance) and decreasing to the zero distance, the volume 118v of the HMD speaker 118 is reduced to zero, and the volume 132v of the detachable speaker unit 132 is slightly reduced to a non-zero value. The controller 150 may cause an audio signal and / or power to be sent to the HMD speaker 118 to output sound therefrom.

[0111] In a fifth operation 750 , while the detachable speaker unit 132 is coupled to the head mounted display unit 110 , the detachable speaker unit 132 continues to provide sound output to the user at a volume 132v greater than zero, while the HMD speaker 118 does not provide any sound to the user.

[0112] In a sixth operation 760 , acoustic transparency of the detachable speaker unit 132 may be activated as previously described with respect to the sixth operation 670 of the method 600 .

[0113] In a seventh operation 770 , noise cancellation of the detachable speaker unit 132 may be activated as previously described with respect to the seventh operation 670 of the method 600 .

[0114] It should be noted that various aspects of the method 600 may be incorporated into the method 700, in which case the volume 118v of the HMD speaker 118 may be changed to simulate the movement of the detachable speaker unit 132 by sound emitted therefrom. For example, in the second operation 720, the volume 118v may be increased to simulate the detachable speaker unit 132 being moved closer to the coupling position on the head mounted display unit 110 and may be increased or kept constant in the third operation 730. In addition, the method 700 may be used in combination with the method 500 described above, for example, in the third operation 530.

[0115] Reference FIG. 8A to FIG. 8B For a detachable speaker unit 132 configured to output sound when detached from the head mounted display unit 110, the equalization of the sound output by the detachable speaker unit 132 may be changed according to the distance from the detachable speaker unit 132 to the head mounted display unit 110 (e.g., the coupling position thereon). For example, as the detachable speaker unit 132 is moved closer to the head mounted display unit 110, the gain of low frequency sounds (e.g., in the bass range) may be increased, and the gain of higher frequency sounds may be reduced. Reducing the gain of low frequencies and increasing the gain of higher frequencies focuses the power output on higher frequency sounds, which generally consume less power than low frequency sounds for a given volume output. Since the power output of the detachable speaker unit 132 may be limited due to the portable / detachable nature (e.g., being battery powered) and / or the main usage scenario (e.g., being kept very close to the user's ear E), equalization in this manner allows the user to experience louder sound when the detachable speaker unit 132 is at a relatively far distance than when equalization is not performed.

[0116] See also Fig. 8A , the gain of the low frequency sound output by the detachable speaker unit 132 is referred to as the low frequency gain 832L, and the gain of the higher frequency sound output by the detachable speaker unit 132 is referred to as the high frequency gain 832H. The low frequency gain 832L and the high frequency gain 832H are represented by a graph in which the Y-axis is the gain and the X-axis is the distance between the detachable speaker unit 132 and the head mounted display unit 110 (e.g., the coupling position thereon), where the zero distance is on the right side (i.e., the detachable speaker unit 132 moves closer to the coupling position moving to the right along the graph).

[0117] At a relatively long distance between the detachable speaker unit 132 and the head mounted display unit 110, the low frequency gain 832L is less than one, for example, starting from zero. As the distance decreases, for example, when the user moves the detachable speaker unit 132 toward the head mounted display unit 110 (for example, to a coupling position above the user's ear E), the low frequency gain increases, for example, ending at one. In contrast, the high frequency gain 832H is higher than one at a relatively long distance and decreases as the distance decreases, for example, ending at one. Therefore, at a relatively long distance, the low frequency gain 832L is lower than one and the high frequency gain 832H is higher than one, and the difference between the low frequency gain 832L and the high frequency gain 832H decreases as the distance decreases, for example, the difference ends at zero.

[0118] Conversely or in addition, at a relatively close distance, such as when the detachable speaker unit 132 is in the user's hand and is being moved toward the head mounted display unit 110 to be coupled, the gain of the low frequency sound may be increased to be above one at a relatively close distance. Such low frequency sounds may have a tendency to roll off (e.g., attenuate) when the detachable speaker unit 132 is held by the user, and increasing the gain of such low frequency sounds may provide the user with a lower sound or the perception of a lower sound. Thus, again, see Fig. 8A , an alternative gain for the low frequency sound output by the detachable speaker unit 132 is referred to as a low frequency gain 832L'. At a relatively close distance between the detachable speaker unit 132 and the head mounted display unit 110, the low frequency gain 832L' may increase to be higher than one, and decrease in a gradual (as shown) or step-wise manner as the distance decreases, such as decreasing toward one and / or decreasing to one. At a relatively far distance, the low frequency gain 832L' may be the same as the low frequency gain 832L (e.g., starting at less than one, such as starting at zero, and increasing as the distance decreases) or other values ​​(e.g., one).

[0119] Figure 8B A method 800 for outputting sound from a detachable speaker unit 132 using balanced audio signal processing according to the position of the audio earphone 130 and / or the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., the coupling position on the head mounted display unit 110) is shown. The method 800 may be implemented, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker unit 132. Although discussed in conjunction with only one of the detachable speaker units 132, the method 800 should be understood to be applicable to each of the detachable speaker unit 132 and the audio earphone 130. The method 800 may be referred to as an equalization method or a balanced audio signal processing method, and may be used alone and / or in conjunction with the method 700.

[0120] In a first operation 810 , a position of the detachable speaker unit 132 relative to the head mounted display unit 110 (eg, a predetermined coupling position thereon) is determined, which may be performed in the manner described above for the first operation 610 of method 600 .

[0121] In the second operation 820, based on the position of the detachable speaker unit 132 being relatively far away (e.g., greater than a threshold distance) from the head mounted display unit 110, the low frequency gain 832L is lower than one and / or the high frequency gain 832H is higher than one, and sound is output from the detachable speaker unit 132 according to the low frequency gain 832L and / or the high frequency gain 832H. The controller 150 may adjust the low frequency gain 832L and the high frequency gain 832H, for example, as Fig. 8A As shown, and causing audio signals and / or power to be sent to the detachable speaker unit 132 to output sound therefrom according to such adjusted gain. The outputted sound may include auditory content, such as auditory content related to a virtual reality experience or a virtual reality experience.

[0122] In the third operation 830, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in the middle range (e.g., below the threshold distance and above the second threshold) and decreasing, the low frequency gain 832L is increased and / or the high frequency gain 832H is decreased, and the sound is output from the detachable speaker unit 132 according to the (adjusted) low frequency gain 832L and / or the high frequency gain 832H. The controller 150 may adjust the low frequency gain 832L and the high frequency gain 832H, for example, as Fig. 8A as shown, and causes audio signals and / or power to be sent to the detachable speaker unit 132 to output sound therefrom according to such adjusted gain.

[0123] In a fourth operation 840, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in a relatively close range (e.g., below a second threshold) and decreasing, the low frequency gain may optionally be increased to be higher than one, and then may further optionally be decreased toward one as the distance decreases to zero. The controller 150 may adjust the low frequency gain 832L' and cause an audio signal and / or power to be sent to the detachable speaker unit 132 to output sound therefrom according to such gain.

[0124] Reference FIG. 9A to FIG. 9B, for a detachable speaker unit 132 configured to output sound when detached from the head mounted display unit 110, the dynamic range of the sound output by the detachable speaker unit 132 may vary according to the distance of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., the coupling position thereon). Dynamic range generally refers to the volume range between loud sounds and soft sounds, while dynamic range compression refers to audio signal processing that reduces the volume of louder sounds and / or amplifies the volume of softer sounds. For example, dynamic range compression may be applied at a reduced level as the detachable speaker unit 132 is moved closer to the head mounted display unit 110, resulting in a lower dynamic range at a relatively far distance and a higher dynamic range at a lower distance. Applying dynamic range compression in this manner may, for example, allow a user to hear more sounds and / or experience a consistently louder volume from the detachable speaker unit 132 at a relatively far distance.

[0125] See also Fig.9A , the dynamic range 932R of the sound to be output by the detachable speaker unit is shown according to the distance between the detachable speaker unit 132 and the head mounted display unit 110 (e.g., the coupling position thereon). As the distance decreases, less dynamic compression is applied, so that the dynamic range 932R increases.

[0126] Fig. 9B A method 900 for outputting sound from a detachable speaker unit 132 using dynamic range compression audio signal processing according to the position of the audio earphone 130 and / or the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., the coupling position on the head mounted display unit 110) is shown. The method 900 can be implemented, for example, using the controller 150 and various sensors of the head mounted display unit 110 and the detachable speaker unit 132. Although discussed in conjunction with only one of the detachable speaker units 132, the method 900 should be understood to be applicable to each of the detachable speaker unit 132 and the audio earphone 130. The method 900 may be referred to as a dynamic range compression method or a dynamic range compression audio signal processing method, and may be used alone and / or in conjunction with the method 700 and / or the method 800.

[0127] In a first operation 910 , a position of the detachable speaker unit 132 relative to the head mounted display unit 110 (eg, a predetermined coupling position thereon) is determined, which may be performed in the manner described above for the first operation 610 of method 600 .

[0128] In a second operation 920, based on the relatively far position of the detachable speaker unit 132 relative to the head mounted display unit 110 (e.g., greater than a threshold distance), a relatively high level of dynamic range compression is applied to achieve a relatively low dynamic range. The controller 150 may apply dynamic range compression, for example, to achieve Fig.9A The dynamic range 932R shown, and causes the audio signal and / or power to be sent to the detachable speaker unit 132 to output sound therefrom according to such dynamic range. The output sound may include auditory content, such as auditory content related to the virtual reality experience or the virtual reality experience.

[0129] In a third operation 930, based on the position of the detachable speaker unit 132 relative to the head mounted display unit 110 being in the mid-range (e.g., below the threshold distance) and decreasing, the level of dynamic range compression applied is reduced to achieve a relatively high dynamic range. The controller 150 may apply dynamic range compression, for example, to achieve Fig.9A The dynamic range 932R shown, and causes audio signals and / or power to be sent to the detachable speaker unit 132 to output sound therefrom according to such dynamic range.

[0130] Reference FIG. 10A to FIG. 10D , the display system 100 may be configured to display a visual indicator 1060 associated with the audio headset 130. For example, before the detachable speaker unit 132 is coupled to the head mounted display unit 110, the head mounted display unit 110 may provide visual content that includes the visual indicator 1060, such as an animation, icon, or other indicator, that may be spatially proximate to the audio headset 130. The visual indicator 1060 may indicate or otherwise indicate the sound output capabilities of the audio headset 130 and / or provide instructions for its use. Although discussed in conjunction with the display system 100 and its components (e.g., the head mounted display unit 110 and the audio headset 130), reference is made to the accompanying drawings. FIG. 10A to FIG. 10D The concepts and methods discussed are applicable to display systems 200 , 300 , 400 .

[0131] For example, in Fig. 10A In the embodiment of the present invention, the user can view the display screen 114 of the head mounted display unit 110 and see the audio headset 130 in the field of view, either as a real object in a mixed reality experience (e.g., via video or optical interpolation as described above), or as a virtual object in a virtual reality experience, where the audio headset 130 is depicted by the rendered graphics. For illustrative purposes, the audio headset 130 is positioned on a table in the real environment. Fig. 10BAs shown, the display screen 114 may also display a visual indicator 1060 that may be visually associated with the audio headset 130 , for example by being drawn in close proximity thereto, such as by surrounding and / or covering the audio headset 130 .

[0132] The visual indicator 1060 indicates or otherwise communicates the audio capabilities and / or use of the audio headset 130. In one example, the visual indicator is an audio-related icon, such as a sound wave (as shown), an illustration of a conventional headset, or other icon indicating the audio capabilities or use of the audio headset 130. Instead or in addition, the visual indicator 1060 may be animated, such as a sound wave emitted from the audio headset 130. The movement and / or intensity of the visual indicator 1060 (e.g., animation of a sound wave) may be associated with the sound associated therewith, such as by changing appearance according to the sound being output. For example, the visual indicator 1060 may visually pulse according to the beat of the sound or music (e.g., faster or slower according to a faster or slower beat of the sound) or the intensity of an acoustic event (e.g., increasing in size and / or pulses faster consistent with a real or virtual event that is expected to produce a loud and / or intense sound, such as an explosion).

[0133] The visual indicator 1060 may be positioned spatially proximate to (e.g., immediately adjacent to, covering, and / or surrounding) the audio earphone 130 and / or each detachable speaker unit 132. For example, by using sensors of the head mounted display unit 110 (e.g., a camera 116 for video recognition of the audio earphone 130, or a motion sensor 120) and / or sensing electronics 138d of the detachable speaker unit 132 (e.g., a camera, a motion sensor, etc.), the position of the audio earphone 130 may be determined in the manner described above.

[0134] The visual indicator 1060 or variations thereof may indicate or otherwise communicate how to use the audio headset 130, such as by displaying one or more diagrams illustrating how to use the audio headset 130. Fig. 10C, for example, when the detachable speaker unit 132 is not in the user's field of view, the visual indicator 1070 can guide the user to move the detachable speaker unit 132 to the coupling position on the head mounted display unit 110. The visual indicator 1070 includes an illustration 1070a representing the detachable speaker unit 132 and another illustration 1070b representing the coupling position. The illustrations 1070a, 1070b can be displayed in a spatial relationship to each other representing the physical spatial arrangement between the detachable speaker unit 132 and the coupling position on the head mounted display unit 110, thereby conveying to the user the required movement for coupling the detachable speaker unit 132. For example, as shown, the illustration 1070a (i.e., representing the detachable speaker unit 132) is positioned downward and to the right of the illustration 1070b (i.e., representing the coupling position on the head mounted display unit 110), which can indicate that the detachable right speaker unit 132 is forward and lower than the coupling position. The visual indication 1070 may instead or additionally include an illustration (e.g., an arrow), text (e.g., written text), and / or an animation (e.g., showing movement of illustration 1070a toward illustration 1070b) to indicate to the user where to move the detachable speaker unit 132 for coupling to the head mounted display unit 110.

[0135] See also Fig. 10D , a method 1000 for providing visual content using a display system and specifically for providing a visual indicator associated with an audio output device. The display system may be, for example, one of the display systems 100, 200, 300, 400. The method 1000 may be implemented, for example, using a controller 150 and various sensors of a head mounted display unit 110 and a detachable speaker unit 132. For simplicity, the method 1000 is discussed in conjunction with the display system 100, but the method 1000 should be understood to be applicable to the display systems 200, 300, 400. In addition, although generally discussed below with respect to the detachable speaker unit 132, the method 1000 should be understood to be applicable to each of the detachable speaker unit 132, the audio headset 130, or another audio output device such as an external speaker.

[0136] In a first operation 1010, the position of an audio output device (e.g., an audio headset 130, a detachable speaker unit 132, or other audio output device) is determined. The position of the audio output device may be determined, for example, by the controller 150 using sensors of the head mounted display unit 110 (such as by processing video received from the camera 116) or using sensing electronics 138d of the detachable speaker unit 132 (e.g., if the detachable speaker unit 132 is not dependent on being coupled to the head mounted display unit 110 for power), such as by detecting movement toward the head mounted display unit 110 and / or changes in proximity therebetween.

[0137] In the second operation 1020, when the audio output device is in the field of view of the head mounted display unit, a visual indicator 1060 is displayed in spatial proximity to the audio output device. For example, the controller 150 may send a display signal to the head mounted display unit 110 to display the visual indicator 1060 using the display screen 114 according to the position determined in the first operation 1010. As described above, the visual indicator 1060 may visually convey to the user the audio characteristics of the audio output device (e.g., showing an icon or animation similar to a sound wave or an audio device type such as a headset) and / or instructions for using the audio output device (e.g., for coupling the detachable speaker unit 132 to the head mounted display unit 110). The field of view of the head mounted display unit 110 is that portion of the real environment that can be seen by the user at a given time.

[0138] In the third operation 1030, when the audio output device is out of the field of view of the display screen 114, a visual indicator 1070 is displayed to show the physical proximity of the audio output device to the head mounted display unit (e.g., the detachable speaker unit 132 to the coupling position). For example, the controller 150 may send a display signal to the head mounted display unit 110 to display the visual indicator 1070 using the display screen 114 according to the position determined in the first operation 1010. As discussed above, the visual indicator 1070 may include an illustration 1070a representing the detachable speaker unit 132, and an illustration 1070b representing the coupling position in a spatial relationship with the detachable speaker unit 132, the spatial relationship indicating the physical spatial relationship (e.g., physical proximity) of the detachable speaker unit 132 to the coupling position on the head mounted display unit 110. It should be noted that one, the other, or both of the second operation 1020 and the third operation 1030 may be performed.

[0139] In a fourth operation 1040, which may occur simultaneously with the second operation 1020 (i.e., when the audio output device is within the field of view) and / or simultaneously with the third operation 1030 (i.e., when the audio output device is outside the field of view), the virtual auditory content may be output by the HMD speaker 118, which may spatially simulate the sound emitted from the audio output device (e.g., spatially balanced from left to right, and / or changing the volume to represent the left-right orientation and / or distance of the audio output device). The virtual auditory content may correspond to the visual indicator 1060 or the visual indicator 1070, for example, in time coincidence with its display.

[0140] Although method 1000 is discussed as providing both visual indicators 1060 , 1070 , method 1000 may instead include operations for providing only one of visual indicators 1060 , 1070 .

[0141] As described above, method 1000 may be used with audio output devices other than audio earphones 130 or detachable speaker unit 132 associated with display system 100. For example, another audio output device may be an external speaker that cannot be physically coupled to the head mounted display unit.

[0142] Furthermore, the method 1000 may be used in conjunction with the methods 500, 600, 700 discussed above. For example, the visual indicators 1060, 1070 may be displayed simultaneously with the sound output from the HMD speaker 118 and / or the detachable speaker unit 132, as described in the methods 500, 600, 700, 800, 900.

[0143] See also Fig.11 , the controller 150 may have a hardware configuration as shown and described, or other suitable configurations. The controller 150 may include a processor 1151, a memory 1152, a storage device 1153, a communication interface 1154, and a bus 1155 connecting such components or devices for communication therebetween. The processor 1151 is operable to execute computer program instructions and perform operations described by such instructions. For example, the processor 1151 may be a central processing unit or other conventional processing device. The memory 1152 may be a volatile, high-speed, short-term information storage device (e.g., a random access memory module). The storage device 1153 may be a non-volatile information storage device (e.g., a hard disk). The communication interface 1154 is configured to send and / or receive signals (e.g., wired or wirelessly), such as sending signals to the head-mounted display unit 110 and / or the audio headset 130 and / or receiving signals from the head-mounted display unit 110 and / or the audio headset 130.

[0144] The physical environment refers to the physical world that people can sense and / or interact with without the help of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through vision, touch, hearing, taste, and smell.

[0145] In contrast, a computer generated reality (CGR) environment refers to a fully or partially simulated environment that people sense and / or interact with via electronic systems. In CGR, a subset of a person's physical movements, or representations thereof, are tracked, and in response, one or more features of one or more virtual objects simulated in the CGR environment are adjusted in a manner that complies with at least one law of physics. For example, a CGR system may detect a person's head turns, and in response, adjust the graphical content and sound field presented to the person in a manner similar to the way such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to features of virtual objects in a CGR environment may be made in response to representations of physical movements (e.g., voice commands).

[0146] A person may sense and / or interact with a CGR object using any of their senses, including vision, hearing, touch, taste, and smell. For example, a person may sense and / or interact with an audio object that creates a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, an audio object may enable audio transparency that selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, a person may sense and / or interact only with an audio object.

[0147] Examples of CGR include virtual reality and mixed reality.

[0148] A virtual reality (VR) environment refers to a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes a plurality of virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing avatars of people are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through a simulation of the person's presence within the computer-generated environment, and / or through a simulation of a subset of the person's physical movements within the computer-generated environment.

[0149] In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, a mixed reality (MR) environment refers to a simulated environment designed to include sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtuality continuum, a mixed reality environment is anything between a fully physical environment at one end and a virtual reality environment at the other end, but not including both ends.

[0150] In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. In addition, some electronic systems used to render the MR environment can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment or representations thereof). For example, the system can cause motion so that virtual trees appear stationary relative to the physical ground.

[0151] Examples of mixed reality include augmented reality and augmented virtuality.

[0152] An augmented reality (AR) environment refers to a simulated environment in which one or more virtual objects are superimposed on a physical environment or a representation thereof. For example, an electronic system for presenting an AR environment may have a transparent or translucent display through which a person can directly view the physical environment. The system may be configured to present virtual objects on a transparent or translucent display so that a person uses the system to perceive virtual objects superimposed on a physical environment. Alternatively, the system may have an opaque display and one or more imaging sensors that capture images or videos of a physical environment that are representations of the physical environment. The system combines the image or video with the virtual object and presents the composition on an opaque display. People use the system to indirectly view the physical environment via an image or video of the physical environment and perceive virtual objects superimposed on the physical environment. As used herein, a video of a physical environment displayed on an opaque display is referred to as a "transparent video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting an AR environment on an opaque display. Further alternatively, the system may have a projection system that projects virtual objects into a physical environment, for example as a hologram or on a physical surface, so that a person using the system perceives the virtual objects superimposed on the physical environment.

[0153] An augmented reality environment also refers to a simulated environment in which a representation of a physical environment is transformed by computer-generated sensory information. For example, in providing a pass-through video, the system may transform one or more sensor images to apply a selected perspective (e.g., a viewpoint) that is different from the perspective captured by the imaging sensor. For another example, a representation of a physical environment may be transformed by graphically modifying (e.g., enlarging) portions thereof so that the modified portions may be representative but not true versions of the original captured images. For another example, a representation of a physical environment may be transformed by graphically eliminating portions thereof or blurring portions thereof.

[0154] An augmented virtual (AV) environment refers to a simulated environment in which a virtual or computer-generated environment is combined with one or more sensory inputs from a physical environment. The sensory input may be a representation of one or more features of the physical environment. For example, an AV park may have virtual trees and virtual buildings, but the faces of people are realistically reproduced from images taken of physical people. For another example, a virtual object may take the shape or color of a physical object imaged by one or more imaging sensors. For another example, a virtual object may take a shadow that conforms to the position of the sun in the physical environment.

[0155] There are many different types of electronic systems that enable people to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, head-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on people's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without tactile feedback), smart phones, tablets, and desktop / laptop computers. A head-mounted system can have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system can be configured to accept an external opaque display (e.g., a smart phone). A head-mounted system can incorporate one or more imaging sensors for capturing images or videos of a physical environment, and / or one or more microphones for capturing audio of a physical environment. Instead of an opaque display, a head-mounted system can have a transparent or translucent display. A transparent or translucent display can have a medium through which light representing an image is directed to a person's eyes. The display can utilize digital light projection, OLED, LED, uLED, silicon-based liquid crystal, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a hologram medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. A projection-based system can employ retinal projection technology that projects a graphic image onto a person's retina. The projection system can also be configured to project a virtual object into a physical environment, such as a hologram or onto a physical surface.

[0156] As described above, one aspect of the present technology is to collect and use data from various sources to display visual content and output audio content, which may include content that changes based on the movement or condition of the user's head or eyes. The present disclosure contemplates that in some instances, these collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0157] The present disclosure recognizes that the use of such personal information data in the disclosed technology can be used to benefit users. For example, personal information data can be used to output content. Therefore, the use of such personal information data enables users to enhance the user experience. In addition, the present disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used to provide insights into the user's overall health status, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0158] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

[0159] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, in the case of observing the condition or movement of a user's head or eyes, the present technology may be configured to allow the user to choose to "opt in" or "opt out" at any time during or after registration for the service to participate in the collection of personal information data. In another example, the user may choose not to provide user movement or condition data, or may choose to limit the length of time for observing or maintaining user observation data. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0160] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0161] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the disclosed technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be provided to a user by inferring preferences based on reasoning based on non-personal information data or an absolute minimum measure of personal information (such as content requested by a device associated with the user or publicly available information).

Claims

1. A display system, comprising: a head mounted display unit that outputs visual content to a user; an audio output device that outputs auditory content to a user; and a controller, the controller controlling the head mounted display unit to output the visual content and controlling the audio output device to output the auditory content; Wherein the visual content output by the head mounted display unit includes a visual indicator that is spatially proximate to the audio output device.

2. The display system according to claim 1, wherein: The visual indicator indicates to a user the sound output capabilities of the audio output device; wherein the position of the audio output device is determined by the controller based on a sensor of one or more of the head mounted display unit or the audio output device; and Wherein, the audio output device is a detachable speaker unit detachably coupled to the head-mounted display unit.

3. The display system according to claim 1, wherein: The visual indicator indicates to a user the sound output capabilities of the audio output device.

4. A display system according to any one of the preceding claims, wherein: The visual indicator includes one or more of surrounding or covering the audio output device in the field of view of the head mounted display unit.

5. A display system according to any one of the preceding claims, wherein: The visual indicator includes a depiction of one or more of sound waves or audio headphones.

6. A display system according to any one of the preceding claims, wherein: The visual indicator is animated.

7. The display system according to claim 6, wherein: The visual indicator is animated by changing appearance in accordance with the sound of the auditory content output by the audio output device.

8. The display system according to any one of claims 1 to 7, wherein: The visual indicator increases in size in response to the sound.

9. The display system according to any one of claims 1 to 7, wherein: The visual indicator visually pulses according to the beat of the sound.

10. The display system according to any one of claims 1 to 7, wherein: The position of the audio output device is determined by the controller based on one or more sensors of one or more of the head mounted display unit or the audio output device.

11. The display system according to any one of claims 1 to 7, wherein: When the audio output device is in the field of view of the head mounted display unit, the field of view is a portion of the real environment that can be viewed by the user through one of video transmission or optical transmission.

12. The display system according to any one of claims 1 to 7, wherein: The audio output device is a detachable speaker unit detachably coupled to the head mounted display unit.

13. A display system comprising: a head mounted display unit that outputs visual content to a user; an audio output device that outputs auditory content to a user and is detachably coupled to the head mounted display unit; and a controller, the controller controlling the head mounted display unit to output the visual content and controlling the audio output device to output the auditory content; The controller controls the head mounted display unit to output visual content to indicate a position of the audio output device relative to the head mounted display unit.

14. The display system of claim 13, further comprising an audio headset, the audio headset comprising the audio output device and another audio output device detachably coupled to the head mounted display unit; in, The visual content includes a graphical representation of the audio output device; wherein the visual content includes instructions for moving the audio output device to the head mounted display unit, the instructions including one or more of an image, text, or animation; and The controller determines the position of the audio output device relative to the head-mounted display unit based on a sensor of one or more of the head-mounted display unit or the audio output device.

15. The display system according to claim 13, wherein: The visual content includes a coupling location of the audio output device and a graphical representation of the audio output device.

16. The display system according to claim 15, wherein: The graphical representation of the coupling location and the audio output device are depicted in spatial relationship relative to each other based on the position of the audio output device relative to the head mounted display unit.

17. The display system according to claim 13, wherein: The visual content includes instructions for moving the audio output device to the head mounted display unit, the instructions including one or more of an image, text, or an animation.

18. The display system of claim 13, further comprising an audio headset comprising the audio output device and another output device detachably coupled to the head mounted display unit.

19. The display system according to claim 18, wherein: The controller determines a position of the other audio output device relative to the head mounted display unit based on a sensor of one or more of the head mounted display unit or the audio output device.

20. A method for providing visual content using a display system, the display system having a head-mounted display unit for outputting visual content to a user, an audio output device for outputting auditory content to the user, and a controller for controlling the head-mounted display unit to output the visual content and the audio output device to output the auditory content, the method comprising: determining, using one or more sensors of one or more of the head mounted display unit or the audio output device and the controller, a position of the audio output device relative to the head mounted display unit; and Based on the position, a visual indicator of spatial proximity to the audio output device is displayed using the head mounted display unit.

Images