Systems and methods for spatial audio enabling safe headphone use during movement and commuting

Abstract

A method for generating sound through one or more wearable audio output devices having a right ear portion and a left ear portion is provided, the method comprising: emitting sound through the wearable audio output devices in a first mode; detecting one or more conditions surrounding a user wearing the one or more wearable audio output devices; determining an activity being performed by the user based on the detected conditions; determining, using one or more processors, whether the determined activity triggers an audio spatialization mode for the one or more wearable audio output devices in which audio output is adjusted with respect to angular movement relative to a virtual audio source; and activating the audio spatialization mode when the determined activity triggers the audio spatialization mode.

Description

[0001] This application is a divisional application of the invention patent application with the application date of February 20, 2019, application number 201980015804.X, and invention name “Spatial audio for safe headphone use during exercise and commuting”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is a continuation-of-patent application of U.S. patent application No. 15 / 908,206, filed on February 28, 2018, the disclosure of which is incorporated herein by reference. Background Art

[0004] Using headphones while cycling, running, or doing other forms of commuting or exercising can be dangerous. For example, using such headphones can interfere with a user's ability to hear their surroundings. This can cause the user to be unaware of objects around or approaching them, potentially leading to collisions or other unexpected events. Audio passing through headphones uses a microphone to record ambient noise, which is then digitally replayed through the headphones with minimal delay, allowing the user to "tune in" and "tune out" of their surroundings. However, such systems don't offer much flexibility and require the user to remember to "tune in." Summary of the Invention

[0005] The present disclosure provides an audio spatialization technology that associates spatialization with identified activities. For example, it can detect when a user is engaging in an activity such as exercising, occupying an area with high traffic volume, etc. Upon detecting such activity, the user's headphones can be switched to an audio-safe spatialization mode, in which sound is generated through the headphones based on their position relative to the location of the sound source. For example, while a user can hear a moving vehicle behind them equally in both ears when facing directly forward, when the user turns their head to the left, they can hear the moving vehicle more clearly in their left ear and more faintly in their right ear.

[0006] One aspect of the present disclosure provides a method for producing sound through headphones having a right ear portion and a left ear portion. The method may include establishing or defining a virtual audio source at a first position, the first position being a first point in space relative to a neutral position of the headphones when a user wears the headphones. Sound is produced through the right ear portion and the left ear portion of the headphones so that the sound is produced with the effect of emanating from the virtual audio source. Angular movement of the headphones relative to the first position is detected so that the angle of the headphones relative to the first position can be determined. Based on the determined angle, the sound level in at least one of the right ear portion or the left ear portion is reduced. When one or more conditions are met, the virtual audio source is moved to a second position, the second position being a second point in space that is different from the first point in space but is the same position relative to the neutral position of the headphones.

[0007] Another aspect of the present disclosure provides a system. The system may include headphones including a left-ear audio output and a right-ear audio output; a memory storing instructions for generating audio through the left-ear audio output and the right-ear audio output; and one or more processors in communication with the processor. The one or more processors are configured to: establish or define a virtual audio source at a first location, the first location being a first point in space relative to a neutral position of the headphones when a user wears the headphones; and generate sound through the right-ear audio output and the left-ear audio output such that the sound appears to emanate from the virtual audio source. The one or more processors are further configured to: detect angular movement of the headphones relative to the first location; determine an angle of the headphones relative to the first location as a result of the detected angular movement; and reduce a sound level in at least one of the right-ear audio output and the left-ear audio output based on the determined angle. Furthermore, the one or more processors may move the virtual audio source to a second location when one or more conditions are met, the second location being a second point in space that is different from the first point in space but is the same location relative to the neutral position of the headphones.

[0008] Another aspect of the present disclosure provides a non-transitory computer-readable medium storing instructions executable by one or more processors in a pair of headphones for performing a method for audio-safe spatialization, the pair of headphones including a left ear portion and a right ear portion. Such a method may include: establishing or defining a virtual audio source at a first location, the first location being a first point in space relative to a neutral position of the headphones when a user wears the headphones; generating sound through the right ear portion and the left ear portion of the headphones so that the sound is generated as if emanating from the virtual audio source; detecting angular movement of the headphones relative to the first location; determining an angle of the headphones relative to the first location as a result of the detected angular movement; reducing a sound level in at least one of the right ear portion or the left ear portion based on the determined angle; and moving the virtual audio source to a second location when one or more conditions are met, the second location being a second point in space that is different from the first point in space but is the same location relative to the neutral position of the headphones.

[0009] Another aspect of the present disclosure provides a method comprising: detecting a direction of movement of an earphone having a right ear portion and a left ear portion; and establishing or defining a virtual audio source for the earphone based on the direction of movement. Sound is generated through the right ear portion and the left ear portion of the earphone so that the sound is generated with the effect of emanating from the virtual audio source. The method further comprises: detecting an angular movement of the earphone relative to the virtual audio source; and adjusting the sound generated through at least one of the right ear portion or the left ear portion based on the angular movement relative to the virtual audio source. In some examples, detecting the direction of movement of the earphone comprises detecting a forward linear motion when the earphone is in a neutral orientation, and establishing the virtual audio source comprises establishing the virtual audio source at a position in front of the earphone. The method may further comprise: detecting a change in the direction of movement of the earphone; and moving the virtual audio source based on the detected change in the direction of movement.

[0010] Another aspect of the present disclosure provides a system comprising: headphones comprising a left-ear audio output and a right-ear audio output; one or more sensors; a memory storing instructions for generating audio through the left-ear audio output and the right-ear audio output; and one or more processors in communication with the memory and the one or more sensors. The one or more processors are configured to: receive input from the one or more sensors; detect a direction of movement of the headphones based on the received input; establish or define a virtual audio source for the headphones based on the direction of movement; generate sound through the right and left ear portions of the headphones so that the sound is generated as if it were emitted from the virtual audio source; detect angular movement of the headphones relative to the virtual audio source; and adjust the sound generated through at least one of the right ear portion or the left ear portion based on the angular movement relative to the virtual audio source. In some examples, the one or more sensors include at least one of an accelerometer, a gyroscope, or a magnetometer.

[0011] More specifically, another aspect of the present disclosure provides a method for generating sound through one or more wearable audio output devices having a right ear portion and a left ear portion, the method comprising: emitting sound through the wearable audio output device in a first mode; detecting one or more conditions surrounding a user wearing the one or more wearable audio output devices; determining an activity being performed by the user based on the detected conditions; determining, using one or more processors, whether the determined activity triggers an audio spatialization mode for the one or more wearable audio output devices, in which audio output is adjusted with respect to angular movement relative to a virtual audio source; and activating the audio spatialization mode when the determined activity triggers the audio spatialization mode.

[0012] The sound produced with the effect emitted from the virtual audio source can be unrelated to the sound of the headset surrounding environment. In particular, the sound produced can be independent of and / or different from one or more or all sound sources (e.g., vehicles) in the headset surrounding environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a schematic diagram of an example system according to aspects of the present disclosure.

[0014] Figure 2 is a block diagram of an example system according to aspects of the present disclosure.

[0015] Figures 3A-3D Example audio spatialization techniques according to aspects of the present disclosure are shown.

[0016] Figures 4A-4C Example audio spatialization techniques according to aspects of the present disclosure are shown.

[0017] Figure 5 is a flowchart illustrating an example method according to aspects of the present disclosure.

[0018] Figure 6 is a flowchart illustrating another example method according to aspects of the present disclosure. DETAILED DESCRIPTION

[0019] Overview

[0020] The present technology is intended to produce sound through earbuds or headphones in a way that improves the user experience and increases safety. In particular, in some cases, the sound is spatialized so that when the user turns his head, the sound appears to be generated from the same position in space as before the user turned his head. If the user's head remains in the position after the turn, the spatialized sound will return to the initial position relative to the user's head. For example, if the sound appears to be initially generated from a position in front of the user, when the user turns to the left, the sound will remain in the same virtual position, appearing to be coming from the right side of the user. If the user keeps turning to the left, the virtual position of the sound will slowly return to the position in front of the user, so that after a period of time, the sound will again appear to be generated from the position in front of the user. The situation where the sound is spatialized may be, for example, when the user is moving, such as during exercise, commuting, etc.

[0021] The technology consists of three synergistic subsystems: 1) an activity detection unit; 2) an audio-safe spatialization algorithm; and 3) a head-mounted inertial measurement unit (IMU). These subsystems can be implemented as an integral part of the headset and / or in conjunction with a communicatively coupled smart device (such as a tablet, watch, phone, etc.).

[0022] An activity detection unit determines whether the user is engaging in an activity that qualifies for activation of the Safe Spatialization mode. Examples of such activities may include bicycling, running, skateboarding, or any other activity in which the user is moving in a given direction within a threshold speed range. For example, the threshold speed range may encompass speeds associated with walking, skating, or bicycling, but may be slower than speeds associated with traveling in a motor vehicle. In some examples, the activity that activates the audio spatialization mode may be an activity in which the user is moving in or near a path traveled by a motor vehicle, other people, animals, etc. The activity may be detected by the earphones themselves, by a paired device (e.g., a phone, a watch, etc.), or by a combination thereof. When such activity is detected, the audio safe spatialization algorithm is activated. In some examples, the audio safe spatialization feature may be activated by user input, such as a gesture via the press of a button. For example, if a user is sitting at a desk working but wants to be able to turn around to better hear a conversation behind them, the user may activate the audio spatialization feature.

[0023] The Audio-Safe Spatialization algorithm reproduces audio as a “virtual audio source” that floats in front of the user. When the user turns his head, the virtual audio source remains in its position relative to the user, which greatly improves the user's spatial awareness of the environment. As an example, if the user is riding a bicycle or running along the street, the virtual audio source will remain “in front of” him (i.e., in the direction he is moving). If the user turns his head to the left to look back (e.g., to check for a car), the volume of the source in his left ear will be significantly reduced, which enables enhanced awareness of sounds emanating from the direction he is facing. In contrast, with a pair of traditional headphones, if the listener turns his head, the volume in both ears remains constant, so the user is equally unable to hear surrounding sounds no matter which direction he is facing. When the Audio-Safe Spatialization feature is not activated, the headphones can operate like a pair of traditional headphones.

[0024] The "virtual audio source" can maintain its position relative to the user's direction of travel. For example, if the user, cycling or jogging, turns a corner in the above example, the virtual audio source can move to a new position ahead of the user in the new direction of travel, as opposed to simply turning their head. The movement of the virtual audio source can be based on time, the detected direction of travel, or some combination of these or other factors.

[0025] When the audio-safe spatialization feature is activated, an angle θ corresponding to the current direction the user is facing is initialized to zero. Angle θ can be determined by an IMU, for example, using data from a gyroscope, magnetometer, accelerometer, etc. The headphones operate with the audio spatialization feature based on this initial value of θ. Data can be continuously acquired from the IMU, such as every 20ms, 200ms, etc. Each time data is acquired from the IMU, angle θ is updated based on the direction the user is currently facing. In some examples, a "zeroing" function can be applied to angle θ with a time constant τ so that the angle of the source relative to the user is always zeroed. Zeroing repositions the virtual audio source of the audio to a position in front of the user over time, such as when the user turns in one direction or another rather than just quickly turning their head. The audio spatialization feature is updated, and the headphones operate based on the current value of θ.

[0026] Example System

[0027] Figure 1 and Figure 2 An example system 100 for audio safety spatialization is shown. The system 100 may include a pair of headphones 180 that communicates with one or more computing devices 160, 170. The computing devices 160, 170 may be, for example, mobile phones, smart watches, gaming systems, tablets, or any other mobile computing devices. In some examples, the example system 100 may be completely contained within the headphones 180, such that the headphones 180 perform all audio safety spatialization operations without communicating with other devices. In other examples, such as Figure 1-Figure 2 As shown, the computing devices 160, 170 can perform some operations and provide information to the headset 180 based on such operations. Figure 1-Figure 2 The headset 180 is shown as communicating with and utilizing information from one or two computing devices, but it is understood that the headset 180 can receive information from any number of computing devices.

[0028] The earphones 180 may include any type of audio source having a left speaker portion 187 and a right speaker portion 188, such as Figure 2As shown. For example, the headphones can be: a pair of earbuds, where a portion of the device is inserted into the concha of the user's ear; a pair of circumaural or over-the-ear speakers, such as held in position covering or around the user's ear by a headband portion; in-ear monitors, which are inserted into the ear canal of the user's ear; or any other type of headphones that provide sound to the user's ear. In some examples, the headphones 180 can include an input portion 185 adjacent to an outer surface, attached to a cable, or elsewhere on the headphones. For example, the input portion 185 can be pressed or otherwise activated to change the operating mode of the headphones 180 or perform other functions. In some examples, the input portion 185 can be activated to switch the headphones 180 between an audio-safe spatialization mode and a standard operating mode.

[0029] The headset 180 can communicate with the computing devices 160, 170 via a wired or wireless connection. For example, the devices can be wirelessly coupled via a short-range pairing connection such as Bluetooth. Other types of wireless connections are also possible. The headset 180 and the computing devices 160, 170 can utilize standard communication protocols such as Ethernet, WiFi, HTTP, protocols described in IEEE 802.11, one or more company-proprietary cellular technologies (such as GSM, CDMA, UMTS, EV-DO, WiMAX, LTE, etc.), and various combinations of the above protocols. The subject matter described herein is not limited to any particular manner of information transmission.

[0030] like Figure 2 As shown, the headset 180 may include one or more processors 281, memory 282, and other components typically found in a headset or other computing device.

[0031] The memory 282 of the headset 180 can store information accessible to the one or more processors 281, including instructions 283 executable by the one or more processors 281. The memory 282 can also include data 284 that can be retrieved, manipulated, or stored by the processor 281. The memory 282 can be of any non-transitory type capable of storing information accessible to the processor 281, such as a hard drive, memory card, ROM, RAM, etc.

[0032] Instructions 283 may be any set of instructions (such as machine code) to be executed directly by one or more processors or any set of instructions (such as a script) to be executed indirectly by one or more processors. In this regard, the terms "instructions," "application," "steps," and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by a processor, or in any other computing device language, including scripts or independent source code modules that are interpreted on demand or compiled in advance. The functions, methods, and routines of the instructions will be explained in more detail below.

[0033] Data 284 can be retrieved, stored, or modified by one or more processors 281 according to instructions 283. The subject matter described herein is not limited to any particular data structure. Data can also be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. In addition, data can include any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other storage (such as in other network locations), or information used by functions to calculate relevant data.

[0034] The one or more processors 281 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor may be a dedicated component, such as an application specific integrated circuit ("ASIC") or other hardware-based processor.

[0035] Headset 180 may also include a transmitter / receiver 286, an output 287, and one or more sensors 288. Transmitter / receiver 286 may be of any conventional type capable of communicating with other devices such as computing device 160. Output 287 may include audio speakers including a first speaker designated for the user's right ear and a second speaker designated for the user's left ear. In some examples, other types of outputs may also be provided, such as tactile feedback, image output on a display, and the like.

[0036] One or more sensors 288 can support the determination of an activity or condition associated with the user. For example, the sensors 288 can include a gyroscope that detects the inertial position of the headset 180, an accelerometer that detects the linear motion of the headset, a magnetometer, a pedometer, a global positioning system (GPS), a camera, a microphone, or any of many other types of sensors. The one or more sensors 288 can operate independently or in conjunction. Based on input from the sensors 288, the processor 281 can determine that the user is, for example, jogging, bicycling, skateboarding, walking on a crowded sidewalk, occupying another busy space, riding public transportation, or engaging in any of many other activities in which the user may want or need to increase awareness of their surroundings. In addition to or in lieu of receiving information from the sensors 288, the one or more processors 281 can receive information from the computing device 160, for example, via the transmitter / receiver 286.

[0037] Instructions 283 may support transitioning to an audio-safe spatialized mode based on input from sensors 288 and / or 167. For example, in a normal or default mode of operation, earphones 180 may produce sound equally and consistently through the right and left ear portions of output 287. However, upon detection of a condition or activity, such as exercise, commuting, or occupying a busy space, instructions may support transitioning to an audio-safe spatialized mode. In this mode, sound is produced through the right and left ear portions of output 287 with the effect of emanating from a virtual audio source at a first location. The first location may be a location relative to the earphones or a user wearing the earphones. For example, the first location may be in front of the user, as shown below in Figure 3A When angular motion of the earphones is detected, the effect of the sound is adjusted based on the angular motion. For example, when the user turns his head to the left, the virtual audio source in front of the user will be output more faintly or quietly in the left ear speaker section and more clearly or loudly in the right ear speaker section. In this regard, based on the adjusted output in the corresponding speaker section, the user perceives the virtual audio source as remaining in the same fixed position in space regardless of the user's adjusted head position. Figure 3B This is further described.

[0038] According to some examples, when one or more conditions are met, the virtual audio source can move to a second position. The second position can be the same as the first position relative to the user or the headset (e.g., in front of the user), but can be a different position in space, such as in combination with Figure 3C As further described, conditions that may result in moving the virtual audio source may include, for example, the headset or the user's head remaining in the adjusted angular position for a predetermined period of time. Such a period of time may be, for example, hundreds of milliseconds, a few seconds, or any other length of time indicating that the user's head is likely to remain in the updated position.

[0039] Computing device 160 may include components similar to those in headset 180, including one or more processors 161 in communication with memory 162 containing instructions 163 and data 164. Computing device 160 may be a mobile computing device intended for use by a user and capable of wirelessly exchanging data with a server over a network, such as the Internet. By way of example only, computing device 160 may be: a mobile phone; or a device such as a wireless-enabled PDA, tablet computer, netbook, smartwatch, head-mounted computing system, or any other device capable of sharing information with headset 180. The user may input information using a keypad, keypad, microphone; using visual signals via a camera; or using a touch screen, among other means.

[0040] The computing devices 160, 170 may have all the components typically used in conjunction with a mobile computing device, such as a processor, memory for storing data and instructions (e.g., RAM and an internal hard drive), a display such as display 165, and a user input device 166 (e.g., input buttons, a touch screen, a microphone, etc.). The client computing device may also include one or more sensors 167 for detecting conditions around the mobile device. For example, the sensors 167 may include an image capture device, such as a camera for recording a video stream and / or capturing images, a speaker, a network interface device, and all components for connecting these elements to each other. The sensors 167 may also include a location determination system, such as a GPS. Other examples of location determination systems may determine location based on wireless access signal strength, images of geographic objects such as landmarks, semantic indicators such as light or noise levels, and the like.

[0041] By way of example and not limitation, sensors 167 may also include gyroscopes, accelerometers, magnetometers, GPS, cameras, microphones, light sensors, motion sensors, and the like. Such sensors can be used to detect a variety of conditions, such as movement, weather conditions, sounds, nearby objects, electromagnetic fields, and the like. In some examples, sensors 167 can identify detailed information related to the condition. For example, gyroscopes and accelerometers can not only detect whether computing device 160 is moving, but also determine whether the movement is linear or rotational, the direction of movement, the speed of movement, and the like. Information from different types of sensors 167 can also be used to determine conditions or activities. For example, a low reading from a light sensor combined with a high reading from a barometer can indicate rain. As another example, a specific speed and vibration level can indicate that computing device 160 is traveling on a bicycle, while received sounds or smells can indicate that computing device 160 is passing a restaurant.

[0042] Memory 162 may store information accessible to one or more processors 161, including instructions 163 executable by one or more processors 161. Memory 162 may also include data 164 that may be retrieved, manipulated, or stored by processor 161. Memory 162 may be of any non-transitory type capable of storing processor-accessible information, such as a hard drive, memory card, ROM, RAM, DVD, CD ROM, memory with write capabilities, and read-only memory.

[0043] Instructions 163 may be any set of instructions to be executed directly by one or more processors (such as machine code) or any set of instructions to be executed indirectly by one or more processors (such as a script). In this regard, the terms "instructions," "application," "steps," and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by a processor, or in any other computing device language, including scripts or independent source code modules that are interpreted on demand or compiled in advance. The functions, methods, and routines of the instructions will be explained in more detail below.

[0044] Data 164 can be retrieved, stored, or modified by one or more processors 161 according to instructions 163. For example, although the subject matter described herein is not limited to any particular data structure, the data can be stored in computer registers, in a relational database as a table with many different fields and records, or in an XML document. The data can also be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. In addition, the data can include any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other storage (such as in other network locations), or information used by functions to calculate the relevant data.

[0045] The one or more processors 161 can be any conventional processor, such as a commercially available CPU. Alternatively, the processor can be a dedicated component, such as an application-specific integrated circuit ("ASIC") or other hardware-based processor. Although not required, the computing device 160 can include dedicated hardware components to perform specific computing processes, such as image recognition, object recognition, speech recognition, etc.

[0046] Instructions 163 may be executed by processor 161, for example, to perform a method comprising: receiving a policy for reporting information; detecting a surrounding condition or activity; determining, based on the policy, whether the detected condition or activity triggers reporting of information, and reporting the information if triggered. Reporting may include at least one of sampling data or uploading information, or both. In some examples, a first condition or activity may trigger sampling of data, while a second, different condition or activity may trigger uploading of information.

[0047] The computing device 160 also includes an input / output (I / O) interface 169. The I / O interface 169 can support communication between the computing device 160 and other devices or networks using analog or digital modulation. The I / O interface 169 can facilitate circuit switching and / or packet switching communication. For example, the I / O interface 169 may include a chipset and antenna arranged for wireless communication with a radio access network or access point. The I / O interface 169 may include a wired interface, such as an Ethernet, a universal serial bus (USB), or a high-definition multimedia interface (HDMI) port. The I / O interface 169 may also include a wireless interface, such as a Wi-Fi or wide-area wireless interface (e.g., WiMAX or 3GPP Long Term Evolution (LTE)). Other forms of physical layer interfaces and other types of communication protocols may also be used. In addition, the I / O interface 169 may include multiple physical communication interfaces (e.g., a Wifi interface, a short-range wireless interface, and a wide-area wireless interface), and may enable multiple simultaneous connections of different types.

[0048] I / O interface 169 can be used to establish a connection with one or more other computing devices, such as headset 180. I / O interface 169 can be used to establish a connection with an access point to detect beacons or radio frequency identification (RFID) tags to determine connection information, etc. Such information can also be used to determine conditions or activities associated with computing device 160, headset 180, or the user. For example, many other computing devices moving in close proximity to computing device 160 can indicate that computing device 160 is on a public train or a busy sidewalk. When combined with other sensor information (such as noise level, vibration, speed, etc.), a more accurate determination can be made.

[0049] While several examples of how various types of detected information may be used, alone or in combination, to determine conditions or activities around computing device 160 are provided, it should be understood that various inputs may be used to determine any number of various activities or conditions. Furthermore, while examples of information or activities detected by one device (such as computing device 160) are described, it should be understood that such information or activities may similarly be detected by other devices (such as headset 180).

[0050] Figure 3A-3C An example of audio spatialization is shown. Figure 3AIn FIG, user 315 is wearing headphones that include a left ear portion 387 and a right ear portion 388. Audio is reproduced through the left ear portion 387 and the right ear portion 388 as a virtual audio source 350 in front of the user. For example, user 315 will perceive audio (such as music, an audiobook, etc.) produced through the headphones as being produced from the virtual audio source 350 in front of the user. As explained in further detail below, when user 315 is facing forward in a neutral position, sound is played equally through the left ear portion 387 and the right ear portion 388. However, when user 315 turns their head, the virtual audio source 350 remains in the same position. Thus, in portions of the headphones that are closer to the virtual audio source 350, the sound is played louder than in portions of the headphones that are farther away from the virtual audio source 350.

[0051] like Figure 3A As shown, a first plane 360 including both the left ear portion 387 and the right ear portion 388 forms a first angle α with respect to a second plane 365 including the virtual audio source 350 and a center point 389 between the left ear portion 387 and the right ear portion 388. In this example, the angle α is approximately 90 degrees, and a first distance 377 between the virtual audio source 350 and the left ear portion 387 is approximately equal to a second distance 378 between the virtual audio source 350 and the right ear portion 388. Therefore, sound can be produced at the same level by the left ear portion 387 and the right ear portion 388. For example, both parts of the earphone can produce sound at the same volume, frequency, saturation, bass, etc.

[0052] While virtual audio source 350 is in front of the user in this example, in other examples, the virtual audio source can be positioned differently relative to the user. For example, virtual audio source 350 can be behind the user, to the side of the user, or in a position corresponding to the source of ambient sound. Furthermore, it should be understood that the angle of virtual audio source 350 relative to the user or headphones can be measured in any of a number of different ways.

[0053] exist Figure 3B, the position of the earphones has changed relative to virtual audio source 350. Specifically, the user has turned their head approximately 45 degrees to the right. Thus, first plane 360, including the earphones, has changed, and therefore, angle α between first plane 360 and second plane 365 has also changed. Furthermore, first distance 377 is now shorter than second distance 378. In this example, the sound produced by left ear portion 387 can be at a higher level relative to the sound produced by right ear portion 388. For example, to create the effect that virtual audio source 350 remains at the same point in space, the sound corresponding to virtual audio source 350 can be played louder by the closer left ear portion 387 than by the more distant right ear portion 388. Any of a number of audio spatialization techniques can be implemented, such as volume-based panning, head-related transfer function (HRTF) spatialization, and the like.

[0054] Figure 3C Another example of position change is provided. In this example, the user is Figure 3A The initial position of the head is rotated by about 90 degrees. Thus, the angle α in this example is even larger, about 180 degrees. In addition, with respect to Figure 3B , the first distance 377 is even shorter, and the second distance 378 is even longer. Therefore, the sound generated by the left ear portion 387 can be relatively Figure 3B The level in is at a higher level, and the sound produced by the right ear portion 388 can be relative to Figure 3B Thus, the user has an enhanced awareness of the surroundings in the direction he is facing.

[0055] In each of the above examples, the virtual audio source 350 remains at approximately the same point in space despite the change in the user's angular position. In the example where the user is moving forward, the virtual audio source 350 may also travel at the same speed so that it remains at the same linear position relative to the user. In such an example, although the linear position may be continuously updated based on the user's movement, as described above in conjunction with Figure 3A-3C The change in angular position can be detected and used to affect a change in the sound produced by the earphones.

[0056] In the event that the change in angular position is the result of a rapid rotation of the user's head, the user may return to the initial position shortly thereafter. As the user's head rotates, the adjustment of the sound emitted by the left ear portion 387 and the right ear portion 388 may be instantaneous. Figure 3A The sound will return to the initial position of the Figure 3AHowever, in some examples, the change in angular position of the earphones or the user's head can persist for a longer period of time. By way of example only, if the user is jogging and turns right, as opposed to quickly turning their head to check the surroundings behind them to the right, the virtual audio source 350 can move to a different position in space so that it is realigned with its initial position relative to the user.

[0057] Figure 3D Examples of moving virtual audio source 350 are shown, such as when the angular position of the user or the headphones changes for at least a predetermined period of time, when the user changes direction of travel, or when other conditions are met. The predetermined period of time can be, for example, one second, a few seconds, or longer. In other examples, sensors in the headphones and / or a coupled computing device can detect the direction of the user's movement and compare that direction to the orientation of the headphones. In such examples, virtual audio source 350 can be moved in response to detecting a change in the user's direction of travel. It should be understood that other conditions for moving virtual audio source 350 are also possible.

[0058] like Figure 3D As shown, the virtual audio source 350 moves to a position in front of the user, which corresponds to the position relative to Figure 3A 388 relative to the user's initial position. Thus, the first plane 360 passing through the left ear portion 387 and the right ear portion 388 is again angled at approximately 90 degrees relative to the second plane 365 passing through the virtual audio source 350 and the center point of the earphone or the user's head. In this regard, the sound generated from the left ear portion 387 and the right ear portion 388 is again balanced so that the effect from the user's perspective is that the virtual audio source 350 is in front of his face.

[0059] The aforementioned audio-safe spatialization feature can be activated upon detecting a specific condition or activity. For example, the headset and / or coupled computing device can determine based on sensor input that the user is jogging, bicycling, skateboarding, traveling in a path close to moving vehicles, traveling on public transportation, occupying a busy space (such as a crowded sidewalk), or engaging in any of many other activities in which the user may desire increased awareness of their surroundings. Upon detecting such a condition or activity, the headset can automatically switch to audio-safe spatialization mode.

[0060] Figures 4A-4C Another example of audio safety spatialization is shown. In this example, a feature is activated in response to detecting that a user has performed a predetermined cycling activity.

[0061] like Figure 4AAs shown, headphone user 405 is riding a bicycle on bicycle lane 420, alongside road 410. Vehicle 440 is also traveling on road 410 and approaching user 405 from behind. Detailed view 450 shows the relative positions of left ear portion 487 and right ear portion 488 of the user's headphones relative to sound source 445 corresponding to vehicle 440. The user's head is facing forward, with left ear portion 487 and right ear portion 488 generally parallel along a plane perpendicular to the user's direction of travel. When sound source 445 approaches from behind to the user's left and slightly closer to the user's right ear than to the left, the sound from vehicle 440, which appears slightly closer, can be heard in the user's left ear portion than in the user's right ear portion. However, virtual audio source 350 is in front of the user, and each of left ear portion 487 and right ear portion 488 is approximately equidistant from virtual audio source 350. Therefore, sounds, such as music, played through left ear portion 487 are at the same level as sounds played through right ear portion 488.

[0062] exist Figure 4B In Figure 4, user 405 turns their head to the left to see approaching vehicle 440. However, virtual audio source 350 remains in the same position. Therefore, as seen in detailed view 451, as a result of the user's head rotation, right ear portion 488 of the earphones is angled closer to virtual audio source 350 than left ear portion 487. As a result, the sound played through left ear portion 487 is reduced in volume compared to the sound played through right ear portion 488. This allows the user to more clearly hear sound source 445 of vehicle 400, which is closer to the user's left ear than to the right ear.

[0063] exist Figure 4C , user 405 has changed direction and is now traveling along path 421 alongside road 411, which is approximately perpendicular to his previous direction of travel. Therefore, as shown in detail view 452, the orientation of the user's headphones is also relative to Figure 4A-4B However, the headset may determine that the change in orientation is due to a change in the user's direction. For example, such a determination may be based on the headset remaining in the updated orientation for at least a predetermined period of time, a detected change in the user's direction of travel, or other information. Thus, the virtual audio source 350 may be automatically adjusted relative to the updated orientation. Figure 4C As shown, the updated orientation of the virtual audio source 350 relative to the earphones returns to a neutral position such that the right ear portion 488 and the left ear portion 487 are again approximately equidistant from the virtual audio source 350. Figure 4A In this orientation, music or other sounds played through the left and right ear portions of the earphones are played at approximately equal levels.

[0064] As described above, the Audio Safe Spatialization feature can be activated upon detecting a specific activity being performed by a user or conditions surrounding the user. However, in other examples, the Audio Safe Spatialization feature can be manually activated by the user, such as by pressing an input button or speaking a verbal command. This way, the user can still benefit from the Audio Safe Spatialization feature while performing other tasks, such as working at a desk in an open office environment, hiking through the woods, shopping, or any other activity that may not necessarily be one of the predetermined activities that would trigger automatic activation of the feature.

[0065] Example Method

[0066] In addition to the operations described above and shown in the accompanying drawings, various operations will now be described. It should be understood that the following operations do not necessarily need to be performed in the exact order described below. Instead, the various steps may be processed in a different order or simultaneously, and steps may also be added or omitted.

[0067] Figure 5 An example method 500 for automatically determining a playback mode for a pair of headphones is shown. The method can be performed, for example, by the headphones. In some examples, the method can also utilize information from one or more computing devices coupled to the headphones (such as a mobile phone, smartwatch, etc.).

[0068] In block 510, the earphones provide audio output in a normal playback mode. For example, in this normal mode, the earphones may provide output at equal levels in both the left ear portion and the right ear portion.

[0069] In block 520, input is received from one or more sensors. Examples of such sensors include accelerometers, gyroscopes, magnetometers, pedometers, GPS, microphones, or any of many other types of sensors. The input may provide information related to the user's movement or the conditions surrounding the user. By way of example only, the information may indicate whether the user is outdoors, the direction the user is moving, the number of people or other devices within a certain proximity of the user, and the like. The input may be received continuously or periodically (such as every few milliseconds, a few tenths of a second, a few seconds, etc.).

[0070] In block 530, the activity performed by the user and / or the conditions surrounding the user may be determined based on the received sensor inputs. For example, it may be determined whether the user is bicycling, jogging, performing some other type of running, commuting, sightseeing, etc.

[0071] In block 540, a determination is made as to whether the identified activity / condition triggers the audio-safe spatialization mode. For example, a set of activities that trigger the audio-safe spatialization mode can be predefined by the manufacturer or by the user. The activities / conditions can be defined with varying degrees of specificity. For example, a more specific activity can specify the user's movements and environment, such as riding a bicycle on a road with more than five vehicles passing per minute. A less specific activity can specify fewer parameters, such as simply jogging. In some examples, threshold parameters can be set to distinguish specific activities from other activities. For example, traveling within a threshold speed range can distinguish riding a bicycle from driving a car.

[0072] If the determined activity / condition does not trigger the audio-safe spatialization mode, the method 500 may return to block 510 where playback continues in normal mode. However, if the activity / condition meets one of the predefined activities / conditions, the headset may automatically switch to operation in the audio-safe spatialization mode (block 550).

[0073] Figure 6 An example method 600 of operation of a headset in an audio-safe spatialization mode is shown.

[0074] In block 610, a virtual audio source is established at a first position relative to a neutral position of the headset. For example, when the user is wearing the headset, an initial angle θ corresponding to the current direction the user is facing is set to zero. The user's orientation can be determined based on information from any one of one or more sensors in the headset and / or a coupled computing device. Examples of such sensors include accelerometers, gyroscopes, magnetometers, etc. The first position can be in front of the user, although in other examples, the first position can be behind the user or at any other relative position.

[0075] In block 620, sound is produced through the left and right ear portions of the headset as if the sound were emitted from a virtual audio source at the first location. Thus, for example, the sound may be played at equal volume in both the left and right ear portions of the headset.

[0076] At block 630, a determination is made as to whether the angular position of the earphone has changed relative to the first position. For example, a determination may be made as to whether the orientation of the earphone has shifted in a manner that indicates the user has turned their head left or right. If the angular position has not changed, method 600 may return to block 620 and continue to generate the indicated sound. However, if a change in angular position is detected, method 600 proceeds to block 640.

[0077] In block 640, an angle of the earphone relative to the first position is determined. For example, where the angle θ is initialized to zero, an updated value of the angle θ may be determined.

[0078] In block 650, the playback in the left and / or right ear portion of the earphone is adjusted based on the determined angle. For example, the volume of the sound in at least one of the left or right ear portion may be reduced. In other examples, the volume of the sound in one of the left or right ear portion may be increased. The amount of volume reduction or increase may vary as a function of the determined angle.

[0079] In some examples, a return-to-zero function can be applied to angle θ so that the angle of the virtual audio source relative to the user always returns to zero. For example, in block 660, a determination is made as to whether a condition for changing the position of the virtual audio source is met. The condition can be based on the amount of time the earphones remain at angle θ, the distance traveled while the earphones are at angle θ, etc. If, for example, the user briefly turns their head to one side and then returns to the initial neutral position, such a condition may not be met. In this case, method 600 returns to block 620. However, if the condition is met, method 600 proceeds to block 670, where the return-to-zero function is applied.

[0080] In block 670, the virtual audio source is moved to a second position in space such that the angular position of the virtual audio source relative to the user is consistent with the initial neutral position. For example, the angle θ may be reinitialized based on the current value determined in block 640. In this regard, the virtual audio source may remain in the same relative position relative to the user for a majority of the user's use of the headset.

[0081] The advantages of the aforementioned systems and methods are that they support increased safety for users wearing headphones while engaging in various activities. By automatically activating the AudioSafe Spatialization feature, users can perform various tasks without having to remember to activate the feature. Furthermore, because the feature enables increased awareness of the user's surroundings, the user will be more prepared to react to their surroundings.

[0082] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. Since these and other variations and combinations of the above-described features may be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be taken as illustrative rather than as limiting the subject matter defined by the claims. Furthermore, the examples provided herein and the use of phrases such as, "such as," "including," and the like should not be construed as limiting the subject matter of the claims to specific examples; rather, these examples are intended to illustrate only one of many possible embodiments. Furthermore, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A method of producing sound through an earphone having a right ear portion and a left ear portion, the method comprising: In the first mode, the sound is emitted through the earphones; detecting information related to movement of a user wearing the headset or conditions surrounding the user; determining an activity being performed by the user based on the detected information; determining, with one or more processors, whether the determined activity triggers an audio-safe spatialization mode for the headset in which audio output is adjusted with respect to angular movement relative to a virtual audio source; as well as When the determined activity triggers the audio safe spatialization mode, activating the audio safe spatialization mode, Determining whether the determined activity triggers the audio safety spatialization mode includes determining whether the activity includes at least one of the following: moving in a given direction within a threshold speed range; Travelling in the path of moving vehicles or on public transport; or Move in the path of others.

2. The method according to claim 1, wherein The threshold speed range includes speeds associated with at least one of walking or bicycling.

3. The method according to claim 1, wherein The threshold speed range is slower than speeds associated with traveling in a motor vehicle. 4 . The method of claim 1 , further comprising detecting angular movement of the earphone when an audio safe spatialization mode is activated.

5. The method of claim 1 , wherein activating the audio-safe spatialization mode comprises: establishing a virtual audio source at a first location, the first location being a first point in space relative to a neutral position of the earphones when the earphones are worn by a user; producing sound through the right ear portion and the left ear portion of the earphones so that the sound is produced as if emanating from a virtual audio source; detecting angular movement of the earphone relative to a first position; determining an angle of the headset relative to the first position due to the detected angular movement; adjusting a sound level in at least one of the right ear portion or the left ear portion based on the determined angle; as well as When one or more conditions are met, the virtual audio source is moved to a second position, the second position being a second point in space that is different from the first point in space but is the same position relative to the neutral position of the headphones.

6. The method of claim 5, wherein adjusting the sound level comprises: reducing the sound level in the left ear portion when the detected angular movement of the earphone is to the left; as well as The sound level in the right ear portion is reduced when the detected angular movement of the earphone is to the right.

7. The method of claim 5, wherein adjusting the sound level comprises: increasing the sound level in the right ear portion when the detected angular movement of the earphone is to the left; as well as The sound level in the left ear portion is increased when the detected angular movement of the earphone is to the right.

8. A system comprising: Headphones, including left and right ear audio outputs; one or more sensors; a memory storing instructions for generating audio via a left-ear audio output and a right-ear audio output; as well as one or more processors in communication with the memory and the one or more sensors, the one or more processors being configured to: emitting sound through the wearable audio output device in a first mode; detecting information related to movement of a user wearing the headset or conditions surrounding the user; determining an activity being performed by the user based on the detected information; determining whether the determined activity triggers an audio-safe spatialization mode for the headset in which audio output is adjusted with respect to angular movement relative to a virtual audio source; and activating the audio safe spatialization mode when the determined activity triggers the audio safe spatialization mode, Determining whether the determined activity triggers the audio safety spatialization mode includes the one or more processors being further configured to determine whether the activity includes at least one of the following: moving in a given direction within a threshold speed range; Travelling in the path of moving vehicles or on public transport; or Move in the path of others.

9. The system according to claim 8, wherein: The threshold speed range includes speeds associated with at least one of walking or bicycling.

10. The system according to claim 8, wherein: The threshold speed range is slower than speeds associated with traveling in a motor vehicle.

11. The system of claim 8, wherein the one or more processors are further configured to detect angular movement of the earphone when an audio-safe spatialization mode is activated.

12. The system of claim 8, wherein upon activating the audio-safe spatialization mode, the one or more processors are further configured to: establishing a virtual audio source at a first location, the first location being a first point in space relative to a neutral position of the earphones when the earphones are worn by a user; producing sound through the right ear portion and the left ear portion of the earphones so that the sound is produced as if emanating from a virtual audio source; detecting angular movement of the earphone relative to a first position; determining an angle of the headset relative to the first position due to the detected angular movement; adjusting a sound level in at least one of the right ear portion or the left ear portion based on the determined angle; as well as When one or more conditions are met, the virtual audio source is moved to a second position, the second position being a second point in space that is different from the first point in space but is the same position relative to the neutral position of the headphones.

13. The system of claim 12, wherein adjusting the sound level comprises: reducing the sound level in the left ear portion when the detected angular movement of the earphone is to the left; as well as The sound level in the right ear portion is reduced when the detected angular movement of the earphone is to the right.

14. The system of claim 12, wherein adjusting the sound level comprises: increasing the sound level in the right ear portion when the detected angular movement of the earphone is to the left; as well as The sound level in the left ear portion is increased when the detected angular movement of the earphone is to the right.

Images