Coordinated tracking for binaural audio playback

Abstract

A method for adjusting an audio output to a listener, comprising: Receiving, from a reference device (106), reference orientation data corresponding to a reference direction of the reference device (106), wherein the reference device (106) is a laptop computer, a tablet computer, a mobile device or a portable computer or an on-board computer of a motor vehicle or aircraft, and wherein the reference direction is a forward direction; Receiving, from a head-mounted device (108) worn by the listener, device orientation data (504) corresponding to a device direction (704) of the head-mounted device (108); outputting, by the head-mountable device (108), the audio output to reproduce a virtual sound source in an original source direction at an offset angle from the device direction; Determining, based on the reference orientation data, whether the head-mountable device (108) is in i) a static application case in which a reference angular change of the reference direction is within a predetermined range of motion, or ii) a dynamic application case in which the reference angular change of the reference direction is outside the predetermined range of motion; and Adjusting, by the head-mounted device (108), the audio output based on the determined use case such that the virtual sound source is reproduced in a set source direction offset from the original source direction by the amount of the reference angle change if the head-mounted device (108) is determined to be in the dynamic use case.

Description

[0001] This application claims the benefit of priority from U.S. Provisional Patent Application No. 62 / 399,250. BACKGROUND area

[0002] Embodiments relating to binaural sound reproduction systems are disclosed. In particular, embodiments relating to binaural sound reproduction systems with head-mounted devices in communication with electronic devices are disclosed. Background information

[0003] Binaural headphones simulate virtual sound sources. To achieve realistic virtual sound sources, head tracking can be used to anchor the virtual sound source to a reference frame, such as a room. Head tracking systems can incorporate orientation sensors that allow an audio engine to predict the orientation of the binaural headphones relative to the reference frame and thus simulate the virtual sound source in an appropriate direction as a listener's head rotates. SUMMARY

[0004] Existing binaural headphones with head tracking can achieve realistic virtual sound sources when the reference frame is stationary. That is, current binaural headphones assume that the virtual sound source is spatially anchored to a stationary reference frame, and thus, movements of the head tracker are attributed to a rotating head of the listener. However, such an assumption may not be appropriate when the reference frame is a moving reference frame or when the listener's entire body is moving relative to the forward-facing direction. For example, the assumption may be incorrect when the listener is jogging along winding city streets or when the listener is moving in the cabin of a car or an airplane. When the reference frame and the user's head experience similar motion, e.g.If an aircraft yaws right from an old heading to a new one, causing a passenger's head to also turn right, a realistic virtual sound source should be positioned in a similar direction relative to the new heading, rather than remaining fixed relative to the old heading. It goes without saying that this does not occur in existing binaural headphones because, as perceived by a listener, the motion imparted to the head tracker by the turning aircraft will result in a shift of the virtual sound source in a leftward direction, even if there is no change in orientation between the listener's head and the moving cabin.

[0005] US 2011 / 0 293 129 A1 relates to a head tracking system that determines a rotation angle of a user's head relative to a reference direction that is dependent on a user's movement. The head tracking system comprises a detection device for measuring head movement to provide a measure representing the head movement, and a processing circuit for deriving the rotation angle of the user's head relative to the reference direction from the measure. The reference direction used in the processing circuit is dependent on the user's movement.

[0006] US 5 373 857 A relates to a head tracker for a virtual reality headset for determining the orientation of the headset relative to the earth's magnetic field, comprising a magnetic sensor responsive to the earth's magnetic field and arranged on the headset and generating, with respect to a vertical axis of rotation of the headset, a displacement signal related to the angular displacement of the headset with respect to a calibration orientation relative to the earth's magnetic field, and a signal processor connected to the magnetic sensor and responsive to the electrical displacement signal for generating an output signal proportional to the orientation of the headset with respect to the calibration orientation.

[0007] The present disclosure is therefore based on the object of specifying a method and a binaural sound reproduction system which are each suitable for enriching the state of the art.

[0008] In one embodiment, a binaural sound reproduction system performs a method for dynamically recentering a reference frame for a virtual sound source. The binaural sound reproduction system includes a reference device having a reference sensor for outputting reference orientation data, and a head-mountable device having a device sensor for outputting device orientation data. The reference orientation data corresponds to a reference direction of the reference device, and the device orientation data corresponds to a device direction of the head-mountable device. Accordingly, the binaural sound reproduction system is provided with system orientation data that can be used to recenter a reference frame of the head-mountable device.

[0009] In a particular embodiment, the head-mounted device includes an audio processor configured to output an audio output to reproduce a virtual sound source in a source direction at an offset angle from a forward device direction. Accordingly, a user of the head-mounted device may perceive the virtual sound source as coming from the source direction. The virtual sound source may be dynamically shifted according to a use case of the head-mounted device. More specifically, the audio output may be adjusted based on a particular use case. Accordingly, the audio processor may be configured to determine, based on the reference orientation data, whether the head-mounted

[0010] Device is in a static application case, e.g. when a reference angle change of the reference direction is within a predetermined range of motion, or in a dynamic application case, e.g. when the reference angle change is outside the predetermined range of motion.

[0011] In one embodiment, when the head-mounted device is in a static use case, a reference frame of the head-mounted device is manually recentered. For example, the head-mounted device may include a recentering input switch to receive recentering input from a user. The audio processor may adjust the audio output in response to receiving the recentering input, e.g., in response to the user pressing a button to recenter the reference frame of the head-mounted device. More specifically, the audio processor may play the virtual sound source in a set source direction at an offset angle from a current forward device direction.The offset angle can be the same angle by which the virtual sound source was previously offset from an initial forward-facing device direction before the user turned their head. Accordingly, the virtual sound source can be manually shifted by the user in a static use case.

[0012] In one embodiment, when the head-mounted device is in a dynamic use case, a reference frame of the head-mounted device is automatically recentered according to a dynamic time constant. To implement automatic recentering, an audio processor may determine an amount of device angle change of a device direction of the head-mounted device, e.g., a degree a user's head rotates. If the amount of device angle change is greater than a predetermined angle change threshold, the audio processor may determine a rate of device angle change. The rate may be determined over a predetermined duration. For example, the predetermined duration may be inversely proportional to the amount of device angle change. That is, the predetermined duration may be greater if the amount of device angle change is smaller.In a particular embodiment, if the determined rate is less than a predetermined rate threshold (indicating that the user is now facing a new forward-facing direction), the audio processor may adjust the audio output to play the virtual sound source in a set source direction. The set source direction may be offset from the original source direction by the amount of the device angle change. Accordingly, automatic recentering of the head-mounted device's reference frame based on the head-mounted device's movement may maintain the user's perception that the virtual sound source is coming from the same direction.

[0013] In one embodiment, when the head-mounted device is in a dynamic use case, a reference frame of the head-mounted device is automatically recentered due to movement of the reference device. To implement automatic recentering, the audio processor may determine an amount of reference angle change in a reference direction of the reference device, e.g., when the reference device rotates as a result of the user jogging or driving around a corner. The audio processor may adjust the audio output to play the virtual sound source in a set source direction. In a particular embodiment, the set source direction is offset from the original source direction by the amount of the reference angle change.Accordingly, the recentering of the reference frame of the head-mounted device is based on a movement of the reference device. Coordinated recentering can maintain the user's perception of the virtual sound source as coming from the same direction.

[0014] The above summary does not include an exhaustive list of all aspects of the present invention. The invention is intended to include all practical systems and methods from all suitable combinations of the various aspects summarized above, as well as those disclosed in the detailed description below and expressly recited in the claims filed with the application. Such combinations have certain advantages not specifically recited in the above summary. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a pictorial view of a user consuming audio or video content in a static use case, according to one embodiment. Fig. 2 is a pictorial view of a user consuming audio or video content in a dynamic use case, according to one embodiment. Fig. 3 is a pictorial view of a binaural sound reproduction system, according to one embodiment. Fig. 4 is a block diagram of a binaural sound reproduction system according to one embodiment. Fig. 5 is a graphical representation of alignment data for a binaural sound reproduction system during a static use case and a dynamic use case according to one embodiment. Fig. 6 is a flow diagram of a method for using a binaural sound reproduction system to automatically recenter a reference frame for a virtual sound source, according to one embodiment. Fig. 7 is a pictorial representation of a binaural sound reproduction system used in a static or dynamic use case, according to one embodiment. Fig. 8 is a flow diagram of a method for using a binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source in a static use case, according to one embodiment. Fig. 9A to 9C are pictorial views of a binaural sound reproduction system in a static application according to one embodiment. Fig. 10 is a flow diagram of a method for using a binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source in a dynamic use case, according to one embodiment. Fig. 11 is a pictorial view of a binaural sound reproduction system in a dynamic use case according to one embodiment. Fig. 12 is a graphical view of an angular change of a head-mounted device of a binaural sound reproduction system in a dynamic use case according to one embodiment. Fig. 13A to 13C are pictorial views of a binaural sound reproduction system in a dynamic application according to one embodiment. Fig. 14 is a flow diagram of a method for using a binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source in a dynamic use case, according to one embodiment. Fig. 15A to 15C are pictorial views of a binaural sound reproduction system in a dynamic application according to one embodiment. DETAILED DESCRIPTION

[0015] Embodiments describe a binaural sound reproduction system and methods for using the binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source. The binaural sound reproduction system may include a reference device, such as a laptop computer, a tablet computer, a mobile device, or a wearable computer, and a head-mounted device, such as a headset or headphones. However, the binaural sound reproduction system may include other devices and apparatuses. For example, the head-mounted device may be a non-head-mounted device, e.g., the device may be a motor vehicle speaker system synchronized with a computer worn by a user. Likewise, the reference device may be an on-board computer of a motor vehicle.

[0016] In various embodiments, the description is made with reference to the figures. However, certain embodiments may be practiced without one or more of these specific details, or in combination with other well-known methods and configurations. In the following description, numerous specific details are set forth, such as specific configurations, dimensions, and methods, to ensure a thorough understanding of the embodiments. In other instances, well-known methods and manufacturing techniques are not described in particular detail in order to avoid unnecessarily obscuring the description. Any reference in this specification to "one embodiment" or the like means that a particular described feature, structure, configuration, or characteristic is included in at least one embodiment.Thus, the appearance of the phrase "one embodiment" or the like in various places throughout this specification does not necessarily refer to the same embodiment. Furthermore, the specific features, structures, configurations, or characteristics may be combined in any suitable manner in one or more embodiments.

[0017] The use of relative terms throughout the specification may denote a relative position or direction. For example, "clockwise" may indicate a first direction of rotation about a reference point. Similarly, "counterclockwise" may indicate a second direction of rotation opposite to the first direction of rotation. However, such terms are used to establish relative frames of reference and are not intended to limit the use or orientation of a binaural sound reproduction system to a specific configuration described in the various embodiments below.

[0018] In one aspect, a binaural sound reproduction system includes a head-mounted device for outputting audio representing a virtual sound source in a source direction, and a secondary reference device that remains fixed relative to a reference frame of the virtual sound source. For example, the secondary device may be located on a torso of a jogging listener, or may be a mobile device or laptop computer resting on a console or tray of a moving automobile or aircraft. Thus, the secondary device may have a reference direction that is a current orientation direction relative to some reference. For example, the reference direction may be a forward direction, e.g., a direction in which the listener is walking or a direction in which the car or aircraft is moving.Orientation data from the secondary device can be used to determine whether the head-mounted device is being used in a static or dynamic use case. Accordingly, movements of the head-mounted device can be differentiated from movements of the secondary device based on the particular use case to adjust the audio output in a manner that realistically localizes the virtual sound source as expected by the listener. That is, the virtual sound source can be positioned relative to the reference frame within which the listener is listening, as determined by the reference device, and local head movements can provide auditory cues to achieve the externalization and localization of the virtual sound source in the audio playback.

[0019] Referring to Fig. 1 shows a pictorial view of a user consuming audio or video content in a static use case, according to one embodiment. A static use case 100 may be a case where a local reference frame 102 associated with a user is stationary with respect to a global reference frame 104. The global reference frame 104 may, for example, be the surface of the Earth below the user. In such a case, a reference device 106, such as a mobile device, a tablet computer, or a laptop computer resting on a desk in front of the user, may remain stationary relative to the local reference frame 102 and the global reference frame 104. Similarly, a torso of the user may remain stationary relative to the local reference frame 102 and the global reference frame 104.Accordingly, the movement of a head-mountable device 108 worn by the user may be attributed to the user rotating his or her head rather than to the local reference frame 102 rotating relative to the global reference frame 104.

[0020] Referring to Fig. Figure 2 shows a pictorial view of a user consuming audio or video content in a dynamic use case, according to one embodiment. A dynamic use case 200 may be a case where a local reference frame 102 associated with a user moves relative to the global reference frame 104. The user may be sitting in a seat of a moving vehicle 202. In such a case, the reference device 106 may rest on a console of the vehicle 202, and thus the reference device 106 may remain fixed relative to the local reference frame 102. Similarly, the user's torso may remain fixed relative to the local reference frame 102. However, the local reference frame 102 may move relative to the global reference frame 104 as the vehicle 202 changes directions.For example, when the vehicle 202 is steered to the right, the local reference frame 102 rotates to the right relative to the global reference frame 104. As such, the reference device 106 or the user's torso, which may be attached to the moving local reference frame 102, may also rotate relative to the global reference frame 104.

[0021] Whether the user hears a virtual sound source rendered by the head-mounted device 108 in the static use case 100 or the dynamic use case 200, it is desirable for the virtual sound source to be stable against the user's head movement. That is, the head-mounted device 108 should adjust audio output to render the virtual sound source in an appropriate direction relative to the local reference frame 102. More specifically, it may be desirable to relocate the virtual sound source when the user turns their head, but not when the head rotation results from turning the user's torso. However, a suitable method for relocating the virtual sound source may depend on the use case.For example, in a static use case 100, when the reference device 106 is fixed relative to the global reference frame 104, the user may want to manually recenter a head tracker if the user wants to rotate in their chair to change a forward direction from an old direction, e.g., pointing toward the reference device 106 on a table, to a new direction, e.g., looking out a window. In contrast, in a dynamic use case 200, when the reference device 106 moves relative to the global reference frame 104, the user may want to automatically update the forward direction to avoid the need to constantly provide manual recentering inputs each time they jog or drive around a corner.

[0022] Referring to Fig. 3, a pictorial view of a binaural sound reproduction system according to one embodiment is shown. A binaural sound reproduction system 300 may include the reference device 106 and the head-mounted device 108. The head-mounted device 108 may output audio to reproduce a virtual sound source in a source direction, as perceived by a user listening to the audio output 302. As described below, the reference device 106 may be a secondary device used to provide orientation data corresponding to a direction or movement of the local reference frame 102. A communication link 304 may be established between the reference device 106 and the head-mounted reference device 108 through a wired or wireless connection to communicate audio or orientation data between the devices.

[0023] The reference device 106 may be an electronic device, such as a smartphone device, a tablet computer, a laptop computer, an automobile's onboard computer, etc. That is, the reference device 106 may be any portable device or device that is movable relative to the global reference frame 104. The reference device 106 may include various capabilities to allow the user to access features including, for example, calls, voicemail, music, email, internet browsing, planning, or photos. The reference device 106 may also include hardware to enable the device to have such capabilities. For example, a housing 306 may include an audio speaker, e.g., a microspeaker, to deliver a far-end voice to a near-end user during a call, and a microphone to capture the user's voice during the call.A display 308 may present video content associated with the audio output 302 to the user. Other conventional features are not shown but may, of course, be included in the reference device 106.

[0024] The head-mounted device 108 of the binaural sound reproduction system 300 may be adapted to present audio content to the user. For example, the head-mounted device 108 may be headphones or a headset with a left speaker 310 and a right speaker 312 to deliver the audio output 302 to the user as stereo sound. The audio output 302 may be associated with music files played by a music player application running on a reference device 106 or with a voice at the far end of a call being handled by the reference device 106. The head-mounted device 108 may include a microphone 314 to capture the user's voice during the call. The microphone 314 may also detect user inputs, such as voice-activated commands.Similarly, the head-mountable device 108 may include manual input features, such as a recentering input switch 316, to receive recentering input from the user, as described below.

[0025] Referring to Fig. 4 shows a block diagram of a binaural sound reproduction system according to one embodiment. The reference device 106 may be any of several types of portable devices or appliances with circuitry adapted for specific functionality. Accordingly, the circuitry depicted in the diagram is provided by way of example and not limitation. The reference device 106 may include one or more processors 402 to execute instructions to perform the various functions and capabilities described below. Instructions executed by the one or more processors 402 of the reference device 106 may be retrieved from local storage 404, which may include a non-transitory machine-readable medium.The instructions may be in the form of an operating system program with device drivers and / or an audio playback engine for playing a virtual sound source according to the methods described below. The one or more processors 402 may also retrieve audio data 406 from the memory 404, including audio data associated with a telephone and / or music playback functions controlled by the telephony or music application programs running on the operating system. To perform such functions, the one or more processors 402 may directly or indirectly implement control loops and receive input signals from other electronic components and / or provide output signals to other electronic components.For example, the reference device 106 may receive input signals from alignment devices of the binaural sound reproduction system 300 and output audio signals to an audio speaker and / or to the head-mounted device 108 via the wired or wireless communication link 304. The communication link 304 may include an audio jack connection in one embodiment, however, an audio jack is only one type of possible connector, and other wired connectors may be used. Furthermore, in one embodiment, the reference device 106 and / or the head-mounted device 108 do not include an audio jack and / or wired connection, and the communication link 304 is established solely through a wireless connection.The head-mounted device 108 may process the audio signals to reproduce a virtual sound source, as described below.

[0026] In one embodiment, the electronic circuitry of the reference device 106 includes a reference sensor 408 to output reference orientation data corresponding to a reference direction 702 of the reference device 106. The reference orientation data may be provided to the one or more processors 402 or the memory 404, and the one or more processors 402 may retrieve the reference orientation data from the memory 404. The reference sensor 408 may be one or more of any known orientation sensors, such as accelerometers, magnetometers, gyroscopes, etc. For example, the reference sensor 408 may be an inertial measurement unit (IMU) integrated within the housing 306 of the reference device 106. However, such inertial-based examples are not limiting, and the reference sensor 408 may include non-inertial sensors, such as optical sensors.More specifically, the reference sensor 408 may be an optical sensor of a camera integrated into a robot mapping system, e.g., a simultaneous localization and mapping system. The robot mapping system may be used to develop and provide reference orientation data corresponding to a reference direction of the reference device 106.

[0027] The reference sensor 408 may capture additional information relevant to a use case of the binaural sound reproduction system 300. For example, the reference sensor 408 may include a global positioning system (GPS) sensor to determine whether the reference device 106 is in transition, e.g., on a road or a rail line. Similarly, the reference sensor 408 may include a microphone to receive ambient sounds that may be comparable to signature sound profiles, e.g., ambient sounds from an aircraft engine, to gather further information about a use case context.

[0028] In one embodiment, the head-mounted device 108 includes a device sensor 410 for outputting device orientation data corresponding to a device direction of the head-mounted device 108. The device sensor 410 may be similar to the reference sensor 408. For example, the device sensor 410 may be an inertial sensor or a non-inertial sensor used to detect an orientation of the head-mounted device 108. Furthermore, the device sensor 410 may detect a context of the head-mounted device 108, i.e., information related to a use case of the head-mounted device 108.

[0029] The head-mounted device 108 may store device orientation data from the device sensor 410 in a corresponding memory (not shown), or the device orientation data may be provided directly to an audio processor 412 of the head-mounted device 108. The audio processor 412 may be configured to present the audio output 302 to the user via a left speaker 310 and a right speaker 312. More specifically, the audio processor 412 may provide audio-electrical signals to the speakers such that stereo sound from the speakers represents a virtual sound source in a source direction. The audio data 406 corresponding to the audio output 302 may be received by the audio processor 412 from the reference device 106 via the wired or wireless communication link 304.For example, the audio data 406 may correspond to a video played on the display 308 of the reference device 106.

[0030] The one or more processors 402 of the reference device 106 and / or the audio processor 412 of the head-mounted device 108 may execute an audio playback algorithm to determine the appropriate audio-electrical signals for the left speaker 310 and a right speaker 312 to reproduce the virtual sound source in the appropriate direction. More specifically, the one or more processors 402 or the audio processor 412 may determine a use case of the binaural sound reproduction system 300 and dynamically recenter a reference frame of the binaural sound reproduction system 300 based on information collected by the reference sensor 408 and / or the device sensor 410. The recentering may be performed manually or automatically by the audio processor 412.

[0031] Referring to Fig. Figure 5 shows a graphical representation of orientation data for a binaural sound reproduction system during a static use case and a dynamic use case, according to one embodiment. As described above, the reference sensor 408 of the reference device 106 may output reference orientation data 502. The reference orientation data 502 may correspond to a rotation of the local reference frame 102. During a static use case 100, the reference orientation data 502 indicates that the reference device 106 is stationary. More specifically, a reference direction of the reference device 106, which by convention is initially oriented in a zero-degree direction relative to the global reference frame 104, remains oriented in the zero-degree direction. That is, the reference device 106 does not experience any discernible angular change or rotation in the static use case 100.In contrast, during the dynamic use case 200, the reference orientation data 502 indicates that the reference device 106 is not stationary. More specifically, the reference direction of the reference device 106 deviates from the zero-degree direction and moves relative to the global reference frame 104. That is, the reference device 106 undergoes an angular change or rotation in the dynamic use case 200. The reference orientation data 502 during the static use case 100 may be typical of the user sitting in a bus while the bus is parked at a bus stop, and the reference orientation data 502 during the dynamic use case 200 may be typical of the user sitting in the bus while the bus is moving along city streets.

[0032] The device sensor 410 of the head-mountable device 108 may output device orientation data 504. The device orientation data 504 may correspond to a head azimuth of the user. During the static use case 100, the device orientation data 504 indicates that the head-mountable device 108 is moving relative to the reference device 106. More specifically, a device direction of the head-mountable device 108 changes as the user looks from left to right. More specifically, in the static use case 100, the device direction of the head-mountable device 108 moves relative to both the local reference frame 102 and the global reference frame 104. Similarly, during the dynamic use case 200, the device orientation data 504 indicates that the head-mountable device 108 is moving as the user looks from left to right.The device orientation data 504 during the static use case 100 may be typical of the user looking around for other passengers on a bus while the bus is parked at a bus stop, and the device orientation data 504 during the dynamic use case 200 may be typical of the user looking out the bus windows as the bus moves along city streets. Accordingly, the device orientation data 504 indicates a degree to which the user's head is moving relative to the global reference frame 104, but does not indicate a degree to which the head movement is attributable to movement of the local reference frame 102 in which the user is located.

[0033] A virtual sound source is presented to the user in such a way that the user perceives the sound source as fixed in space. However, maintaining the virtual sound source in a position expected by a listener may require the binaural sound reproduction system 300 to distinguish between movements of the listener's head caused by rotation of the user's neck and movements caused by rotation of the local reference frame 102 in which the user is located. Accordingly, to anchor the virtual sound source to the local reference frame 102, the sound reproduction system 300 may operate by evaluating a use case and recentering a reference frame for the virtual sound source based on the determined use case.

[0034] Referring to Fig. 6 is a flow diagram of a method for using a binaural sound reproduction system to automatically recenter a reference frame for a virtual sound source, according to one embodiment. Fig. Figure 7 is a pictorial representation of a binaural sound reproduction system used during the procedure of Fig. 6 is used. Accordingly, Fig. 6 and Fig. 7 are described together below. At operation 602, the processors of the binaural sound reproduction system 300 may receive reference alignment data 502. The reference alignment data 502 may be output by the reference sensor 408 of the reference device 106.

[0035] Referring to Fig. 7, for example, the reference orientation data 502 may correspond to a reference direction 702 of the reference device 106. The reference direction 702 may be determined by convention. For example, the reference direction 702 may be an output from an IMU or navigation system that corresponds to a reference quantity, such as a vector pointing forward from a top surface of the housing 306. However, the reference direction 702 may not actually point forward from the user 706. That is, the secondary device 106 does not need to be aligned or even know which direction is an actual forward direction. Instead, the determinations described throughout this specification may be based on relative changes in orientation and do not necessarily take into account an actual forward direction.As such, the term "forward-facing" as used throughout the specification is to be interpreted as a relative term and not necessarily an absolute term that takes into account the spatial orientation of the user 706. At operation 604, processors of the binaural sound reproduction system 300 may receive device orientation data 504. The device orientation data 504 may be output by the sensor 410 of the head-mountable device 108.

[0036] Referring to Fig. 7, the device orientation data 504 may correspond to a device direction 704 of the head-mountable device 108. Of course, the head-mountable device 108 may be worn by a user 706, and thus the device direction 704 may correspond to a forward direction of a user 706. Accordingly, the device direction 704 may change relative to the global reference frame 104 when the user 706 turns their head or when the local reference frame 102 within which the user 706 is located moves relative to the global reference frame 104.

[0037] At operation 606, the head-mounted device 108 provides the audio outputs 302. More specifically, the audio processor 412 may generate an electrical audio signal for the left speaker 310 and the right speaker 312 to reproduce a virtual sound source 708 in a source direction 710. The virtual sound source 708 may be associated with content played on the reference device 106. For example, the virtual sound source 708 may be a voice of a participant seated toward the periphery of the user 706 during a video conference call. Accordingly, to accurately represent the virtual sound source 708 to the user 706, the audio output 302 may reproduce the virtual sound source 708 at an offset angle 712 from the device direction 704 such that the voice is perceived by the user 706 as coming from the edge of their view.

[0038] The local reference frame 102 may be movable relative to the global reference frame 104. If the local reference frame 102 moves, the reference device 106, which may be fixed relative to the local reference frame 102, may also move.

[0039] When the reference device 106 rotates, the reference direction 702 may experience a reference angular change relative to a datum of the global reference frame 104, e.g., relative to true north. When the local reference frame 102 moves, the device direction 704, which corresponds to the forward direction of the user 706, may also move. However, to accurately represent the virtual sound source 708, any movement in the device direction 704 due to the movement of the local reference frame 102 should be compensated for by also shifting the source direction 710. That is, when the local reference frame 102 moves relative to the global reference frame 104, the head-mounted device 108 may be recentered so that the virtual sound source 708 continues to come from a direction that the user 706 perceives as the periphery of their view.Such recentering may occur in response to determining an appropriate recentering method, ie, a method based on the use case of the head-mountable device 108.

[0040] At operation 608, the one or more processors 402 and / or the audio processor 412 of the binaural sound reproduction system 300 may determine whether the head-mountable device 108 is in the static use case 100 or the dynamic use case 200. Such a determination may be based on the reference orientation data 502. More specifically, the reference angular change of the reference direction 702 may be compared to a predetermined range of motion 714 to evaluate whether the head-mountable device 108 is being used in the static use case 100 or the dynamic use case 200.

[0041] The range of motion 714 may be an angular range, e.g., -20 to 20 degrees, relative to a base reference direction of the reference device 106. That is, if the reference orientation data 502 indicates that the reference direction 702 experiences a static reference angular change 716 within the range of motion 714, e.g., less than 20 degrees in any direction, the audio processor 412 may determine that the head-mountable device 108 is in the static use case 100. Angular deviations within the range of motion 714 may be attributed to natural displacements within a given static environment, e.g., a torso rotation during walking on a treadmill, and may be insufficient to change the recentering method from a manual method to an automatic method, as described below.

[0042] If the reference orientation data 502 indicates that the reference direction 702 detects a dynamic reference angle change 718 outside the predetermined range of motion 714, e.g., more than 20 degrees in any direction, the audio processor 412 may determine that the head-mountable device 108 is in the dynamic use case 200. Angular deviations outside the range of motion 714 may be attributed to preconceived dynamic environments, e.g., jogging or driving around a corner, and may be sufficient to change the recentering method from a manual method to an automatic method, as described below.

[0043] At operation 610, the binaural sound reproduction system 300 may adjust the audio output 302 based on the particular use case. More specifically, the audio processor 412 of the head-mounted device 108 may alter electrical audio signals provided to the left speaker 310 and the right speaker 312 to reproduce the virtual sound source 708 in an adjusted source direction. Shifting the virtual sound source 708 in the adjusted source direction may be achieved using different techniques. For example, as described below, the virtual sound source 708 may be shifted based on either a manual or automatic recentering of the local reference frame 102 associated with the head-mounted device 108.

[0044] Referring to Fig. 8 is a flow diagram of a method for using a binaural sound reproduction system to automatically recenter a reference frame for a virtual sound source in a static use case, according to one embodiment. Fig. 9A to 9C are pictorial views of the binaural sound reproduction system during the procedure of Fig. 8. Accordingly, Fig. 8 and 9A to 9C are described together below.

[0045] At operation 802, one or more processors of binaural sound reproduction system 300 may determine that head-mountable device 108 is in a static use case 100. Such a determination may be based on a reference angular change of reference direction 702 being within range of motion 714, as described above.

[0046] When the head-mounted device 108 is in the static use case 100, the reference frame of the head-mounted device 108 may be manually recentered. By way of example, the local reference frame 102, as indicated by the reference orientation data 502, may remain stationary relative to the global reference frame 104. Nevertheless, a user 706 may wish to recenter the device direction 704 in a new forward-facing direction. For example, the user 706 may wish to rotate their chair to look out a window while listening to music playback.

[0047] Referring to Fig. 9A, the user 706 may initially point in a first direction, such that the device direction 704 points forward in the static use case 100. As described above, the virtual sound source 708 may be rendered to be perceived by the user 706 as coming from a surrounding direction at an offset angle 712 from the reference direction 702.

[0048] Referring to Fig. 9B, user 706 may rotate their office chair so that a torso and face of user 706 are directed in a second direction that is offset from the first direction. User 706 may desire the second direction to be a new forward direction. More specifically, the second direction may be a current device direction 902 that is offset from device direction 704 by an adjustment angle 904. In Fig. 9B, the virtual sound source 708 may continue to be rendered in source direction 710 because the head-mounted device 108 does not automatically recenter the reference frame of the virtual sound source 708 to accommodate movements of a user's torso in the static use case 100. Thus, although the user 706 has shifted their personal reference frame by rotating their chair and may expect the virtual sound source 708 to shift to match the personal reference frame, the virtual sound source 708 may instead be perceived as coming from nearly the same direction as the user's new gaze.

[0049] At operation 804, the user 706 may manually override the binaural sound reproduction system 300 to override the reference frame of the head-mounted device 108 so that the virtual sound source 708 moves to the expected location.

[0050] Referring to Fig. 9B, a recentering input 906 may be received by the audio processor 412 when the head-mounted device 108 has a current device direction 902. The recentering input 906 may be a manual input from the user 706. For example, the user 706 may actuate the recentering input switch 316 to provide the recentering input 906 to the head-mounted device 108. The recentering input switch 316 may be a voice-activated switch actuated by a verbal command issued by a user 706. Similarly, the recentering input switch 316 may be a physical button on the head-mounted device 108, and the user 706 may manually press the physical button to provide the recentering input 906.

[0051] At operation 806, the audio output 302 may be adjusted in response to determining that the head-mounted device 108 is in the static use case 100 and in response to receiving the recentering input 906. For example, the audio processor 412 may receive the recentering input 906 from the recentering input switch 316 after determining that the head-mounted device 108 is in the static use case 100, and the audio processor 412 may adjust the audio output 302 to play the virtual sound source 708 in a set source direction.

[0052] Referring to Fig. 9C, a set source direction 908 may be at an offset angle 712 from the current device direction 902. Accordingly, the user 706 may manually calibrate a zero-degree direction of the head-mountable device 108 to align with the user's gaze by activating the recenter input switch 316. Thus, the virtual sound source 708 will continue to be perceived as coming from the user's 706 peripheral vision after the user 706 has rotated in their chair and manually recentered the reference frame of the head-mountable device 108.

[0053] Referring to Fig. 10, a flowchart of a method for using a binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source in a dynamic use case is shown, according to one embodiment. An understanding of the Fig. 10 is described with reference to Fig. 11, 12 and 13A to 13C, and accordingly these figures are described in combination below.

[0054] At operation 1002, one or more processors of binaural sound reproduction system 300 may determine that head-mountable device 108 is in a dynamic use case 200. Such a determination may be based on a reference angular change of reference direction 702 that is outside of range of motion 714, as described above.

[0055] When the head-mounted device 108 is in the dynamic use case 200, the reference frame of the head-mounted device 108 may be automatically recentered. By way of example, if the local reference frame 102, as indicated by the reference orientation data 502, moves relative to the global reference frame 104, the binaural sound reproduction system 300 may recenter the device direction 704 to a new forward direction. Accordingly, the virtual sound source 708 may be translated to remain fixed within the moving local reference frame 102, as perceived by the moving user 706. In one embodiment, a manner of automatically translating the virtual sound source 708 may depend on an amount and / or rate of device angular change.

[0056] Referring to Fig. 11 shows a pictorial view of a binaural sound reproduction system in a dynamic use case, according to one embodiment. At operation 1004, an amount and rate of angular change of device direction 704 is determined. Device angle change 1102 may be measured as an angular distance between an initial device direction 704 and a current device direction 902 after the user's head has moved in dynamic use case 200. For example, device angle change 1102 may be 90 degrees if user 706 jogs around a corner and shifts the forward direction from device direction 704 pointing along a street to current device direction 902 pointing along an orthogonal street.

[0057] In one embodiment, the amount of device angle change 1102 may be within various ranges of motion. For example, the device direction of the head-mountable device 108 may move from an initial device direction 704 to a current device direction over an angle within a range of motion, and the range of motion may be one of multiple ranges of motion offset from the initial device direction by at least a predetermined angle change threshold, e.g., a first angle change threshold 1104. The amount of device angle change 1102 may be more than a first angle change threshold 1104. Small head movements made by the user 706 while the head-mountable device 108 is in the dynamic use case 200 may not require the virtual sound source 708 to be translated.The first angle change threshold 1104 may correspond to the predetermined range that includes small head movements and glances that should not cause the virtual sound source 708 to jump. In contrast, it may be desirable to translate the virtual sound source 708 more when the head-mountable device 108 experiences larger device angle changes 1102. Thus, the device angle changes 1102 may be further divided into ranges of motion. A first range of motion may include the range of motion between the first angle change threshold 1104 and a second angle change threshold 1106. A second range of motion may include the range of motion between the second angle change threshold 1106 and a third angle change threshold 1108. A third range of motion may include the range of motion beyond the third angle change threshold 1108.

[0058] The audio processor 412 may determine whether the amount of angular change 1102 of the device is less than a second predetermined angular change threshold 1106, greater than the second angular change threshold 1106 and less than a third angular change threshold 1108, or greater than the third angular change threshold 1108. The processor 412 may determine that the device direction of the head-mountable device has moved from the initial device direction 704 to a current device direction within the first range of motion when the device direction is between the first angular change threshold 1104 and the second angular change threshold 1106.The audio processor 412 may determine that the device direction of the head-mountable device has moved from the initial device direction 704 to a current device direction within the second range of motion when the device direction is between the second angle change threshold 1106 and the third angle change threshold 1108, etc. Recentering may occur based on the angular change within which the current device direction 902 falls. That is, the audio processor 412 may adjust the audio output based on the range of motion of the device direction to play the virtual sound source in an adjusted source direction offset from the source direction traversed by the head-mountable device by the angle 1102.

[0059] Referring to Fig. Figure 12 shows a graphical view of an angular change of a head-mounted device of a binaural sound reproduction system in a dynamic use case according to one embodiment. In one embodiment, the audio processor 412 may determine a rate of device angular change in response to the amount of device angular change 1102 being greater than the first predetermined angular change threshold 1104. For example, the audio processor 412 may determine a rate of device angular change when the device direction moves within the first range of motion between thresholds 1104, 1106 or the second range of motion between thresholds 1106, 1108.When the movements of the head-mounted device 108 are within a small range, the virtual sound source 708 may remain fixed relative to an existing reference frame of the head-mounted device 108. However, when the movements of the head-mounted device 108 exceed the first angle change threshold 1104, the audio processor 412 may begin evaluating when and where to move the virtual sound source 708. Such an evaluation may be made based on a rate 1202 of device angle change within the range of motion.

[0060] The rate of device angle change may be analyzed in terms of angle over time. In one embodiment, a rate 1202 of device angle change corresponds to the amount of device angle change per unit time. The rate of device angle change may be an amount of device angle change over a bin duration. For example, the analyzed time range may be divided into individual bins, and each bin may have a bin duration 1204. Accordingly, the audio processor 412 may determine the rate 1202 of device angle change 1202 over the predetermined bin duration 1204. The rate 1202 may be a median rate of change of the device direction when the device direction is within the given range of motion. For example, if the bin duration 1204 is set to 100 ms, a median rate of change of the device direction may be measured over each 100 ms time window.

[0061] In one embodiment, the bin duration 1204 is based on the amount of device angle change 1102. For example, the bin duration 1204 may correspond to the range of motion within which the head-mountable device is pointed and / or moving. The bin duration 1204 may be a first duration, e.g., 100 ms, if the amount of device angle change 1102 is greater than the first angle change threshold 1104 and less than the second angle change threshold 1106. The bin duration 1204 may be a second duration that is different from the first duration if the amount of device angle change 1102 is greater than the second angle change threshold 1106. For example, if the amount of device angle change 1102 is between the second angle change threshold 1106 and the third angle change threshold 1108, the container duration 1204 may be a different value, e.g., 25 ms.If the magnitude of the device angle change 1102 is greater than the third angle change threshold 1108, the bin duration 1204 may be a different value, e.g., 5 ms. Thus, a length of the bin duration 1204 may be inversely correlated with an magnitude of the device angle change 1102. That is, the second bin duration associated with angle changes greater than the second angle change threshold 1106 (within the second range of motion) may be less than the first bin duration associated with angle changes less than the second angle change threshold 1106 (within the first range of motion).

[0062] At operation 1006, the audio processor 412 may adjust the audio output 302 in response to the rate 1202 of device angle change being less than a predetermined rate threshold 1206. Referring again to Fig. 12, the variance between individual bins can be compared to the rate threshold 1206. For example, the median rates of change of device direction 704 can be analyzed to determine whether the rate 1202 has decreased to a point where it is safe to assume that the user 706 is now facing in a direction that is a new forward-facing direction. It should be understood that rates 1202 of change during extreme movements, such as jogging or turning a corner, may be higher than the rates of change that occur while the user 706 is facing in a forward direction. Accordingly, by changing the bin duration 1204 inversely to the amount of device angle change 1102, appropriate smaller time windows can be analyzed to determine whether the user 706 has turned to face in a new forward-facing direction.That is, large rotations may result in almost instantaneous shifts of the virtual sound source 708, while smaller rotations may shift the virtual sound source 708 more gradually. As a result, adjustments to the source direction 710 of the virtual sound source 708 may more naturally align with movements of the user 706.

[0063] Fig. 13A to 13C are pictorial views of the binaural sound reproduction system during the procedure of Fig. 10. With reference to Fig. 13A, in the dynamic use case 200, the head-mounted device 108 may face the reference direction 702, while the audio output 302 reproduces the virtual sound source 708 in the source direction 710. As described above, the source direction 710 may be offset from the device direction 704 at an offset angle 712. Referring to Fig. 13B, the user 706 may shift the device direction 704 to the current device direction 902 by walking around a corner. More specifically, the device direction 704 may experience a device angle change 1102. The amount of the device angle change 1102 may be greater than the first angle change threshold 1104, indicating to the head-mounted device 108 that the audio output 302 should be adjusted to relocate the virtual sound source 708. Referring to Fig. 13C, in response to the head-mounted device 108 being in the dynamic use case 200 and in response to the rate 1202 of the device angle change 1102 being less than the predetermined rate threshold 1206, as described above, the audio processor 412 may adjust the audio output 302 to play the virtual sound source 708 at an adjusted source direction 908. The angular shift may be equal to the device angle change 1102. That is, after determining that the device angle change 1102 is a result of the user 706 shifting to a desired forward-facing direction, the virtual sound source 708 may be shifted by the amount of the device angle change 1102 to maintain the perception that the virtual sound source 708 is at an equal offset angle 712 from the current device direction 902.

[0064] The methods described throughout this specification do not necessarily require the determination of static use case 100 or dynamic use case 200 by reference device 106 to be useful for head tracking during binaural sound reproduction. For example, one or more of the methods may be performed without making an initial determination as to whether the binaural sound reproduction system is being used in a dynamic case. That is, binaural sound reproduction system 300 may be assumed to be in dynamic use case 200 (or static use case 100), and audio output 302 may be adjusted accordingly.

[0065] In one embodiment, head tracking for binaural sound reproduction includes a method similar to the method of Fig. 10. However, operation 1002 may be omitted. More specifically, the audio output may be continuously updated according to operations 1004 and 1006 without making a determination as to whether the head-mountable device 108 is in the dynamic use case 200. As such, adjustments to the audio output may be based on the Fig. 11 to 12 to determine the processes illustrated in Fig. 13A to 13C without performing an initial determination according to operation 1002. Accordingly, time-based head tracking for binaural sound reproduction may be performed using a dynamic time factor that continuously determines an appropriate source direction 710 (or 908). This is emphasized to clarify that any of the described methods may be performed with fewer operations than described, and indeed, operations from different methods of binaural sound reproduction may be combined within the scope of this description. Accordingly, the described methods are illustrative and not limiting.

[0066] Referring to Fig. 14, a flow diagram of a method for using a binaural sound reproduction system to dynamically recenter a reference frame for a virtual sound source in a dynamic use case is shown, according to one embodiment. Fig. 15A to 15C are pictorial views of the binaural sound reproduction system during the procedure of Fig. 14. Accordingly, Fig. 14 and 15A to 15C are described together below.

[0067] Referring to Fig. 15A, in dynamic use case 200, head-mounted device 108 may face reference direction 702, while audio output 302 reproduces virtual sound source 708 in source direction 710. As described above, source direction 710 may be offset from device direction 704 at an offset angle 712.

[0068] At operation 1402, one or more processors of binaural sound reproduction system 300 may determine that head-mountable device 108 is in a dynamic use case 200. Such a determination may be based on a reference angular change of reference direction 702 that is outside of range of motion 714, as described above.

[0069] When the head-mounted device 108 is in the dynamic use case 200, the reference frame of the head-mounted device 108 may be automatically recentered. By way of example, if the local reference frame 102, as indicated by the reference orientation data 502, moves relative to the global reference frame 104, the binaural sound reproduction system 300 may recenter the device direction 704 to a new forward direction. Accordingly, the virtual sound source 708 may be automatically translated to remain fixed within the moving local reference frame 102, as perceived by the moving user 706.In one embodiment, one way of automatically moving the virtual sound source 708 may include coordination between the reference alignment data 502 from the reference device 106 and the device alignment data 504 from the head-mounted device 108.

[0070] Referring to Fig. 15B, the reference device 106 may experience a reference angle change that causes the reference direction 702 to shift from the initial reference direction 702 to a new reference direction 1502. The new reference direction 1502 may be offset from the reference direction 702 by an adjustment angle 904. In one embodiment, the change in the reference direction 702 may also occur in the device direction 704. For example, if the reference device 106 and the user 706 are both located in a moving vehicle, both the reference device 106 and the head-mounted device 108 will experience the same angular change as the vehicle turns the corner. Accordingly, the device direction 704 may rotate by the adjustment angle 904 from the current device direction 902.

[0071] At operation 1404, the audio processor 412 may determine the magnitude of the reference angle change of the reference direction 702. More specifically, when the reference direction 702 rotates by the adjustment angle 904, the audio processor 412 may determine that the magnitude of the reference angle change is equal to the adjustment angle 904.

[0072] At operation 1406, the audio processor 412 may adjust the audio output 302 based on the amount of reference angle change. More specifically, the audio output 302 may be adjusted in response to determining that the head-mounted device 108 is in the dynamic use case 200 to reproduce the virtual sound source 708 in an adjusted source direction 908 offset from the original source direction 710. The amount of adjustment may be the same as the reference angle change. Accordingly, the virtual sound source 708 may translate in coordination with angular displacements of the reference device 106. That is, if the reference device 106 rotates by an amount, the virtual sound source 708 may translate by the same amount.As a result, the virtual sound source 708 may be automatically shifted to remain fixed within the moving local reference frame 102 as perceived by the moving user 706.

[0073] Referring to Fig. 15C, automatically recentering the reference frame of the head-mounted device 108 relative to the local reference frame 102 associated with the reference device 106 may allow the reference frame of the virtual sound source 708 to shift based on movements of the reference device 106 rather than the movements of the head-mounted device 108. Specifically, after the virtual sound source 708 is shifted to the set source direction 908, the user 706 may rotate their head without affecting a location of the virtual sound source 708 relative to the local reference frame 102. However, the virtual sound source 708 may be rendered differently as the user 706 rotates their head.For example, if the user 706 turns their head to change the device direction 704 from a current direction 902 back to the initial device direction 704, the virtual sound source 708 may still be perceived as coming from the set source direction 908, which may now be at a greater angle from the device direction 704 than before.

[0074] It is understood that the recentering processes described above may be combined in hybrid embodiments. For example, a tuning method for controlling how often a component of the binaural sound reproduction system 300 updates a direction may be applied to the reference device 106. Such a tuning method may be similar to the methods described above with reference to Fig. 10. For example, referring again to Fig. 15B the amount of reference angle change with angle change thresholds similar to those with respect to Fig. 11. In one embodiment, a rate of reference angle change may be determined in response to the magnitude of the reference angle change being greater than a predetermined angle change threshold, e.g., greater than a first angle change threshold as applied to the movement of the reference device 106. The rate of reference angle change may be determined over a predetermined duration. That is, the rate of reference angle change may be determined in a manner similar to the determination of rate 1202, as described with reference to Fig. 12. Accordingly, referring again to operation 1406 of Fig. 14, the adjustment of the audio output 302 may further be made in response to the rate of the reference angle change being less than a respective predetermined rate threshold. Such a procedure is similar to the method used to recenter the head-mountable device 108 based on a gaze direction of the user 706. It will be appreciated that the application of such a smoothing method to the dynamic recentering method of Fig. 14 can provide a further benefit. The application of the smoothing algorithm to the reference device 106 enables the binaural sound reproduction system 300 to accurately and smoothly locate the new reference direction 1502 so that the virtual sound source 708 can then be shifted by an appropriate adjustment angle 904.

[0075] As described above, sensor inputs to the binaural sound reproduction system 300 can be classified into different use cases.

[0076] When the binaural sound reproduction system 300 determines a dynamic use case 200, recentering is used according to one of the methods described above. To illustrate an application of the binaural sound reproduction system 300, consider the case where the user 706 is watching a movie in an airplane. The user 706 may watch the movie using the binaural sound reproduction system 300. For example, the reference device 106 may be a tablet computer, and the display 308 may present video content to the user 706. The head-mounted device 108 may be a pair of headphones whose sound is calibrated such that dialogue from the video content is perceived as coming from the forward direction, i.e., the display 308, while the surround sound content is perceived as coming from behind the user 706. When the user 706 moves their head, e.g.To look out an airplane window, the dialogue is still perceived as coming from the forward-facing direction of the tablet computer. It can be seen that when the airplane yaws, without the aid of a dynamic recentering function, a head tracker would detect the airplane's rotation as a rotation of the user's head, and thus the dialogue and ambient content would be incorrectly rotated. However, using the dynamic recentering processes described above, the binaural sound reproduction system 300 can distinguish between the user's head movement and the movement of the reference frame (airplane) and can compensate to ensure that the dialogue and ambient content are reproduced correctly.

[0077] In the foregoing specification, the invention has been described with reference to certain exemplary embodiments thereof. It will be apparent that various modifications may be made thereto without departing from the broader spirit and scope of the invention as set forth in the following claims. The specification and drawings are accordingly to be considered as illustrative and not restrictive.

Claims

[1] A method for adjusting an audio output to a listener, comprising: Receiving, from a reference device (106), reference orientation data corresponding to a reference direction of the reference device (106), wherein the reference device (106) is a laptop computer, a tablet computer, a mobile device or a portable computer or an on-board computer of a motor vehicle or aircraft, and wherein the reference direction is a forward direction; Receiving, from a head-mounted device (108) worn by the listener, device orientation data (504) corresponding to a device direction (704) of the head-mounted device (108); outputting, by the head-mountable device (108), the audio output to reproduce a virtual sound source in an original source direction at an offset angle from the device direction; Determining, based on the reference orientation data, whether the head-mountable device (108) is in i) a static application case in which a reference angular change of the reference direction is within a predetermined range of motion, or ii) a dynamic application case in which the reference angular change of the reference direction is outside the predetermined range of motion; and Adjusting, by the head-mounted device (108), the audio output based on the determined use case such that the virtual sound source is reproduced in a set source direction offset from the original source direction by the amount of the reference angle change if the head-mounted device (108) is determined to be in the dynamic use case. [2] The method of claim 1, wherein the reference direction is a direction in which the listener is walking or in which the motor vehicle or aircraft is moving. [3] The method of claim 1, further comprising receiving a recentering input if the head-mountable device (108) has a current device direction; and if the head-mountable device (108) is determined to be in the static use case, in response to receiving the recentering input, including playing the virtual sound source in a set source direction at the offset angle from the current device direction (704). [4] The method of claim 3, wherein the recentering input includes manual input from a user. [5] The method of claim 1, further comprising: Determining an amount of device angle change of the device direction; and Determining, in response to the amount of device angle change being greater than a first predetermined angle change threshold, a rate of device angle change over a predetermined duration; wherein adjusting the audio output in response to determining that the head-mountable device is in the dynamic use case and in response to the rate of device angle change being less than a predetermined rate threshold includes playing the virtual sound source in an adjusted source direction offset from the source direction by the amount of device angle change. [6] The method of claim 5, wherein the predetermined duration is based on the amount of device angle change, wherein the predetermined duration is a first duration when the amount of device angle change is less than a second predetermined angle change threshold, wherein the predetermined duration is a second duration when the amount of device angle change is greater than the second predetermined angle change threshold, and wherein the second duration is less than the first duration. [7] The method of claim 1, further comprising determining, in response to the magnitude of the reference angle change being greater than a predetermined angle change threshold, a rate of the reference angle change over a predetermined duration; wherein adjusting the audio output is in response to the rate of the reference angle change being less than a predetermined rate threshold. [8] Binaural sound reproduction system comprising: a reference device (106) having a reference sensor for outputting reference orientation data corresponding to a reference direction of the reference device (106), wherein the reference device (106) is a laptop computer, a tablet computer, a mobile device or a portable computer, or an on-board computer of a motor vehicle or aircraft, and wherein the reference direction is a forward direction; and a head-mountable device including a device for outputting device orientation data corresponding to a device direction of the head-mountable device, and an audio processor configured to Outputting an audio output to reproduce a virtual sound source in an original source direction at an offset angle from the device direction Determining, based on the reference alignment data, whether the head-mountable device is in an i) static application case in which a reference angular change of the reference direction is within a predetermined range of motion, or an ii) dynamic application case in which the reference angular change of the reference direction is outside the predetermined range of motion, and Adjusting the audio output based on the determined use case such that the virtual sound source is reproduced in a set source direction offset from the original source direction by the amount of the reference angle change if the head-mountable device (108) is determined to be in the dynamic use case. [9] A binaural sound reproduction system according to claim 8, wherein the reference direction is a direction in which the listener walks or moves within the motor vehicle or aircraft. [10] The binaural sound reproduction system of claim 9, wherein the head-mountable device (108) further includes a recentering input switch to receive a recentering input from a user when the head-mountable device (108) has a current device direction, and wherein the audio processor is further configured to Receiving the recentering input from the recentering input switch, and if it is determined that the head-mountable device (108) is in the static use case, Adjusting the audio output in response to receiving the recentering input to reproduce the virtual sound source in a set source direction at the offset angle from the current device direction (704). [11] A binaural sound reproduction system according to claim 10, wherein the recentering input switch is a physical button to receive a manual press by the user. [12] A binaural sound reproduction system according to claim 9, wherein the audio processor is further configured to: Determining an amount of device angle change of the device direction (704); and Determining, in response to the amount of device angle change being greater than a first predetermined angle change threshold, a rate of device angle change over a predetermined duration; wherein, in response to determining that the head-mountable device is in the dynamic use case and in response to the rate of device angle change being less than a predetermined rate threshold, the audio processor adjusts the audio output to reproduce the virtual sound source in an adjusted source direction offset from the source direction by the amount of device angle change. [13] The binaural sound reproduction system of claim 12, wherein the predetermined duration is based on the amount of device angle change, wherein the predetermined duration is a first duration when the amount of device angle change is less than a second predetermined angle change threshold, wherein the predetermined duration is a second duration when the amount of device angle change is greater than the second predetermined angle change threshold, and wherein the second duration is less than the first duration. [14] The binaural sound reproduction system of claim 9, wherein the audio processor is further configured to determine a rate of reference angle change over a predetermined duration in response to the magnitude of the reference angle change being greater than a predetermined angle change threshold; wherein the audio processor adjusts the audio output in response to the rate of reference angle change being less than a predetermined rate threshold. [15] A non-transitory, machine-readable medium having instructions that, when executed by a processor of a binaural sound reproduction system, cause the binaural sound reproduction system to perform a method comprising: Receiving, from a reference device (106), reference orientation data corresponding to a reference direction of the reference device (106), wherein the reference orientation data includes a reference angle change of the reference direction, and wherein the reference device (106) is a laptop computer, a tablet computer, a mobile device or a portable computer or an on-board computer of a motor vehicle or aircraft, and wherein the reference direction is a forward direction; Receiving, from a head-mounted device (108) worn by a listener, device orientation data (504) corresponding to a device direction (704) of the head-mounted device (108); outputting, by the head-mountable device (108), an audio output to reproduce a virtual sound source in an original source direction at an offset angle from the device direction (704); Determining a use case of the head-mountable device, wherein the head-mountable device is determined to be in a static use case if the reference angle change is within a predetermined range of motion, and wherein the head-mountable device is determined to be in a dynamic use case if the reference angle change is outside the predetermined range of motion; and Adjusting, by the head-mounted device, the audio output based on the determined use case such that the virtual sound source is reproduced in an adjusted source direction offset from the original source direction by the amount of the reference angle change if the head-mounted device (108) is determined to be in the dynamic use case. [16] The non-transitory machine-readable medium of claim 15, further comprising instructions to cause the binaural sound reproduction system to perform the method, further comprising receiving a recentering input when the head-mountable device has a current device direction; wherein adjusting the audio output in response to determining that the head-mountable device is in the static use case and in response to receiving the recentering input includes playing the virtual sound source in an adjusted source direction at the offset angle from the current device direction. [17] The non-transitory machine-readable medium of claim 15, further comprising instructions for causing the binaural sound reproduction system to perform the method, further comprising: Determining an amount of device angle change of the device direction; and Determining, in response to the amount of device angle change being greater than a first predetermined angle change threshold, a rate of device angle change over a predetermined duration; wherein adjusting the audio output in response to determining that the head-mountable device is in the dynamic use case and in response to the rate of device angle change being less than a predetermined rate threshold includes playing the virtual sound source in an adjusted source direction offset from the source direction by the amount of device angle change. [18] The non-transitory machine-readable medium of claim 15, wherein the reference direction is a direction in which the listener is walking or moving within the motor vehicle or aircraft.

Images