System and method for automatic audio routing between device and periphery

Abstract

The disclosure relates to systems and methods for improving routing of audio output signals from an electronic device to an audio output device. Such a system may include at least one inertial measurement unit (IMU) disposed in an audio output device; and at least one processor configured to receive data from the at least one IMU, determine from the received data a current state of the audio output device is worn or unworn, and if the current state of the audio output device is worn, determine that the audio output device is not worn. The audio output signal is routed from the electronic device to the audio output device if the current state of the audio output device is worn, and the audio output signal is routed to the audio output of the electronic device if the current state of the audio output device is unworn.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to consumer electronic devices, and more specifically to systems and methods for processing and routing audio outputs between an electronic device and one or more peripheral audio output devices communicatively coupled to said electronic device. Background Technology

[0002] Electronic device management is typically accomplished by users manually controlling the device through inputs such as buttons, dial pads, and touch surfaces or touchscreens. This manual control of device features can be cumbersome and detract from the overall user experience when interacting with electronic devices.

[0003] In particular, routing audio output from electronic devices such as smartphones and laptops to peripherals such as headphones, earphones, or earbuds and speakers presents challenges. These challenges are exacerbated for incoming audio or audio / video calls. Many user interfaces present users with several different options, and when an incoming call rings and flashes on the screen, it is difficult for users to quickly identify and select the desired option from these presented choices before the call is transferred to voicemail or the caller hangs up.

[0004] Conventional approaches to addressing these challenges often rely on proprietary solutions, where simplifications or shortcuts may be taken when the electronics manufacturer is also an audio peripheral manufacturer. Other conventional approaches use strict user settings to route calls in an "if / then" manner, without giving the user the opportunity to make different choices on the fly. Summary of the Invention

[0005] Therefore, these shortcomings still need to be addressed.

[0006] The various embodiments disclosed herein aim to address the aforementioned problems, including making it easier for users to select and manage audio outputs by routing audio outputs from electronic devices to one or more desired audio output devices.

[0007] In one embodiment, a system for improving the routing of audio output signals from an electronic device to an audio output device includes: at least one inertial measurement unit (IMU) disposed in the audio output device; and at least one processor configured to: receive data from the at least one IMU, determine from the received data whether the current state of the audio output device is worn or not, route the audio output signal from the electronic device to the audio output device if the current state of the audio output device is worn, and route the audio output signal to the audio output of the electronic device if the current state of the audio output device is not worn.

[0008] In another embodiment, a method for routing an audio output signal from an electronic device to an audio output device includes: acquiring data from at least one inertial measurement unit (IMU) of the audio output device; determining from the acquired data whether the current state of the audio output device is worn or not; if the current state of the audio output device is worn, routing the audio output signal from the electronic device to the audio output device; and if the current state of the audio output device is not worn, routing the audio output signal to the audio output of the electronic device.

[0009] The foregoing overview is not intended to describe every embodiment or implementation of the subject matter of this disclosure. The accompanying figures and detailed description below illustrate various embodiments in more detail. Attached Figure Description

[0010] The subject matter of this disclosure can be more fully understood by considering the following detailed descriptions of various embodiments in conjunction with the accompanying drawings, wherein:

[0011] Figure 1 A set of headphones according to one implementation scheme is described.

[0012] Figure 2A yes Figure 1 A partial view of a pair of headphones.

[0013] Figure 2B yes Figure 2A Detailed view.

[0014] Figure 3A A set of headphones and associated pick-up angle ranges according to one implementation scheme are described.

[0015] Figure 3B A set of headphones and the associated pick-up angle range according to another embodiment are depicted.

[0016] Figure 4 It is a block diagram of a system based on an implementation plan.

[0017] Figure 5 It is a flowchart based on an implementation plan.

[0018] Figure 6 This is a flowchart based on another implementation scheme.

[0019] Figure 7 This is another flowchart based on an implementation plan.

[0020] While various embodiments may have various modifications and alternatives, specific implementations have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that the claims are not intended to be limited to the specific embodiments described. Rather, they are intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the subject matter of this disclosure as defined by the claims. Detailed Implementation

[0021] This disclosure relates to embodiments that improve audio output routing from electronic devices to audio peripherals, such as from smartphones to headphones. Embodiments may also include (or operate in conjunction with) improved and more intuitive user interface features, such as status lights. Both the audio output routing and the user interface features may operate based on or dependent on the detected usage status or condition of the electronic device, audio output device, or both. The embodiments thus provide seamless and intuitive audio output routing without requiring user interaction or selection.

[0022] In the examples discussed herein, electronic devices are typically smartphones, and audio output devices are typically a set of headphones (generally worn on or above the ears). These examples are for convenience and ease of illustration of the various concepts discussed herein and are not limiting with respect to other implementations. For example, electronic devices may also include tablet computers, e-readers, wearable devices (such as smartwatches), laptop computers, game consoles or devices, computers or other computing devices, stereo or streaming audio sources, tuners, “smart” home devices or home automation devices (such as video doorbells), cameras, televisions, home theater systems, home appliances, fitness equipment (such as exercise bikes), vehicles, or some other device capable of providing audio signals to audio output devices—generally referred to as “electronic devices”. Similarly, audio output devices may also include earbuds (generally worn inside the ear), headphones (generally worn on or above the ear), headsets (which may have only one earpiece or headphone unit), clothing or other wearable devices with speakers or headphones embedded therein or coupled thereto (e.g., hats, headbands, helmets, glasses or goggles with integrated speakers or headphones), home furnishings (e.g., chairs, sofas, pillows or cushions with integrated speakers), speakers, portable speaker units, vehicle parts (e.g., headrests or seats with integrated speakers), or virtually any other device to which speakers or headphones may be embedded or to which electronic devices may provide audio signals for output—therefore, generally, referred to as "audio output devices".

[0023] In some implementations, the electronic device and the audio output device may be the same (or integrated), such as in a vehicle, smartphone, gaming device, or system. In other implementations, the electronic device and the audio output device are wirelessly coupled, such as via Bluetooth, Wi-Fi, NFC, or some other wireless communication protocol. In still other implementations, the electronic device and the audio output device are wired coupled.

[0024] Based on this introduction, now refer to Figure 1 The document depicts a set of headphones 100 according to one embodiment. The headphones 100 include a headband 102 that pivotally or flexibly couples headphone units 104 to both ends. The headband 102 is adjustable such that the length of the headband 102 between the headphone units 104 can be shortened, lengthened, or otherwise adjusted to better or more comfortably fit a particular user. The headband 102 may optionally include a padded portion 106 to further enhance user comfort during wear.

[0025] Each headphone unit 104 includes an earcup 108 and an ear pad 110. The earcup 108 houses electronic components, such as speaker drivers and associated circuitry, which are configured to generate sound when the headphones 100 are worn and to project sound into the user's ears within the ear pad 110.

[0026] One or more outer surfaces or portions of each earcup 108 may include at least one input 112 for receiving user input. Input 112 may be one or more of a button, slider, touch-sensitive surface, etc. Input 112 may be configured to receive user input to control, for example, the power, volume, noise cancellation, fit, comfort, noise isolation, and other features of the headset 100. Input 112 may be located anywhere on the headset 100 such that it is available for manual input by the user when worn or as needed.

[0027] In some embodiments, the earcups 108 may further include one or more input ports 114 configured to receive a cable connector and one or more indicators 116, such as light-emitting diodes (LEDs), configured to convey status information of the headset 100. These input ports 114 may include audio ports or jacks, such as USB or mini-jacks as examples. The arrangement of the inputs 112, input ports 114, and indicator lights 116 may vary between the earcups 108 (e.g., left and right) or in different embodiments or versions of the headset 100.

[0028] refer to Figure 2A and 2B According to an implementation plan Figure 2A The internal portion of the headphone unit 104 coupled to the headband 102 is depicted. Figure 2Byes Figure 2A A rotating close-up perspective view of region 212 of the headphone unit 104. The headphone unit 104 includes an earcup 108 configured to contain a speaker driver (not shown), at least one sensor 220, and processing hardware (not shown). As previously described, the headphone unit 104 is coupled to the headband 102 via a coupling mechanism 222, which in this embodiment includes a hinge 224. Other coupling mechanisms may be used in other embodiments.

[0029] In some embodiments, sensor 220 includes an inertial measurement unit (IMU). The IMU may include at least one accelerometer and at least one gyroscope. In other embodiments, the IMU may also include a magnetometer or other sensing devices. In one embodiment, each earphone unit 104 of the headset 100 includes an IMU.

[0030] In operation and reference Figure 3A and 3B The IMU can measure one or more of a specific force, angular velocity, or orientation associated with the headset 100. Therefore, the IMU can have a static or dynamic (e.g., selectable or adjustable) range, such as... Figure 3A and 3B The orientation of the headset 100 can be measured across approximately (e.g., plus / minus 10 degrees) 110 degrees. Figure 3A and 3B In the implementation plan, the 110-degree range begins and ends at approximately 35 degrees to the horizontal.

[0031] In other embodiments, both the range and the angle relative to the horizontal can vary. In some embodiments, any angular change greater than 35 degrees, such as at least 35 degrees, at least 45 degrees, at least 60 degrees, at least 75 degrees, and at least 90 degrees, can be detected quickly (e.g., within about 0.5 seconds, or less than about 1 second). This is done for various reasons, such as to prepare the headset 100 and to indicate the relevant status via indicator light 116.

[0032] In some implementations, the range or angle relative to the horizontal (reference included) Figure 3A and 3B At least one of them may be optional or adjustable. The range and angle discussed here by way of example generally assume an upright or usable position, wherein the headband 102 is worn on the top of the user's head and each earphone unit 104 is worn on the user's ears and is arranged generally vertically.

[0033] Therefore, if the user from Figure 3APick up the headphones 100 in the indicated orientation and hold them at approximately 45 degrees to the horizontal, ready to put them on (i.e., the headband 102 is rotated upwards, generally along...). Figure 3A (As indicated by the direction of the middle arrow), the IMU can detect that the headset 100 has moved or changed orientation. This enables the sensing or detection of the position of the headset 100, including whether the headset 100 is:

[0034] • Place it on a table or other surface.

[0035] • It is held in the user's hand, ready to be worn.

[0036] • Worn on the user's head and above both ears.

[0037] • Worn on the user's head, with only one earcup (108) placed above the ear.

[0038] • Worn around the user's neck.

[0039] • Hanging on a bracket.

[0040] Some of these positions or wearing conditions may also utilize data from additional sensors in the headset 100, as will be discussed in more detail below. Embodiments of this disclosure can visually detect, whether solely through the IMU or taking into account data and measurements from other sensors 220 of the headset 100, when the headset 100 is worn, removed, fully worn, partially worn, held in the hand, etc. This visual detection can be used to activate, deactivate, or alter the operation of one or more features of the headset 100 or coupled electronic devices.

[0041] In Figure 3A In another related example, the orientation of the headset 100 detected by the IMU can be used to turn the headset 100 on or "wake it up". It is assumed that the headset 100 is already oriented in a generally horizontal position (e.g., Figure 3A (As shown) Place the headset 100 on a table or other surface for a period of time. If no change in position is detected after a period of time, the headset 100 may automatically enter a "sleep" mode or shut down completely. In some embodiments, if the user subsequently picks up the headset 100 or tilts the headset 100 from horizontal at least 35 degrees (e.g., causing the headband 102 to rise to a certain angle), the headset 100 may automatically enter a "sleep" mode or shut down completely. Figure 3A If the headset is lifted at a 110-degree angle (as shown in the diagram), it can be woken up or turned on.

[0042] In Figure 3BIn another similar example, it is again assumed that the headphones 100 are placed on a surface and are in off or sleep mode. If the user raises the headphone unit 104 from horizontal position by at least 35 degrees (e.g., ... Figure 3B (As indicated by the arrow in the image) Pick up the headphones 100 so that the headphone unit 104 rises to... Figure 3B (As shown, when lifted at a 110-degree angle), the headset 100 can be woken up or powered on. This movement—raising the headphone unit 104 or tilting the headset 100 so that the headband 102 rotates downwards—is useful for the user to check the indicator 116. The so-called “tilt to check” feature allows the user to tilt or invert the headset 100. When the IMU detects this movement or orientation, the indicator 116 provides status information of the headset 100, such as the illumination color or pattern of one or more LEDs in the indicator 116. A red LED indicates that the headset 100 is powered off. A flashing LED indicates that the headset 100 is in sleep or low-power mode. A white or green LED indicates that the headset 100 is powered on and active. A blue LED indicates that the headset 100 is ready to enter Bluetooth pairing mode.

[0043] In other examples, tilting the headphones 100 in a particular orientation, as described above, can cause the headphones 100 to automatically wake up or turn on. Moving or tilting the headphones in another orientation or manner can cause other activities, such as turning off the headphones 100 or disabling a feature (e.g., pausing audio output). Some of these features (such as automatic on or off) may depend on whether these features are available or activated on the headphones 100, as the activation of some features can be user-selectable.

[0044] These specific examples are not critical, nor is the specific type or arrangement of the indicator 116 critical. The point of these examples is that moving or tilting the headset 100 in various ways or orientations can cause features or states of the headset 100 to be activated, deactivated, or changed in a manner generally intuitive to the user.

[0045] In addition to one or more IMUs providing this activation, deactivation, or alteration of the headphones or related to the headphones, the headphones 100 may also include at least one additional (or alternative) sensor mode in at least one sensor 220. For example, in one embodiment, at least one sensor 220 also includes a force sensor configured to detect at least one characteristic associated with the headphones 100, such as a force exerted by the headband 102 or another object on one or both of the earphone units 104. For example, when the headphones 100 are worn on a user's head, the headband 102 is biased to flex or bend outward to exert a generally inward force (i.e., toward the user's head or ears) on each earphone unit 104. This bias provides a good "seal" of the ear pads 110 around each of the user's ears for a better audio experience (e.g., improved noise cancellation), but not so strong as to affect a comfortable fit.

[0046] In some embodiments, additional sensor modes that may be included in sensor 220 and headset 100 include an accelerometer for sensing acceleration, a gyroscope for sensing orientation, a magnetometer or Hall effect sensor for sensing magnetic fields, a proximity sensor, a capacitive touch sensor for sensing touch, a millimeter (MM) wave sensor for sensing reflected signals indicating the angle, range, and velocity of a sensed object, an infrared sensor for sensing thermal radiation, a temperature sensor for sensing temperature, a humidity sensor for sensing humidity, or one or more other sensors known to those skilled in the art. In embodiments that implement multiple sensor modes, so-called “sensor fusion” may be implemented by or for headset 100. Sensor fusion combines data from multiple sensors or sources, reducing uncertainty in data processing or providing additional information compared to what a single sensor mode might provide individually.

[0047] Regardless of quantity and modality, sensor 220 is communicatively coupled to onboard processing hardware, which may include at least one processor and memory on which firmware / software may reside. In some embodiments, data collected by sensor 220 may alternatively or additionally be communicated to a server or electronic device (e.g., an electronic device such as a smartphone) communicatively coupled to or remote from headset 100.

[0048] Therefore, refer to Figure 4 A block diagram of a system 400 for detecting the usage status of a headset 100 is depicted according to one embodiment. System 400 includes a headset 100 and electronics 430. The headset 100 generally includes a processor 440, memory 442, one or more sensors 220, one or more IMU sensors 444, one or more proximity sensors 446, and at least one power supply 448.

[0049] System 400 can be used to determine the usage state of the headset 100 (e.g., wearing for active use vs. not wearing vs. partially wearing), which can be used to configure one or more features or operations of the headset 100. As discussed elsewhere herein, the features and concepts of this disclosure discussed in examples related to the headset 100 can be used in or applied to other devices, such as in-ear headphones and other electronic devices with user interface features, for which determining the usage state and configuring or controlling one or more features or operations based on the determined usage state may be desirable and helpful. Therefore, in various embodiments, the headset 100 may alternatively be any electronic device incorporating one or more headphone units, the electronic device comprising at least in-ear earbuds or on-ear headphones. The headset 100 may alternatively include various wearable electronic devices, such as virtual reality (VR) headsets, augmented reality (AR) headsets, smartwatches, smart glasses, smart jewelry or other smart accessories, gaming devices, medical or health devices, or other electronic devices worn or used on or near the user's body or body parts.

[0050] Processor 440 (and any other processor, processing device, or engine discussed herein) can be any programmable device that accepts digital data as input, is configured to process the input according to instructions or algorithms, and provides results as output. In one embodiment, processor 440 can be a central processing unit (CPU) or a microcontroller or microprocessor configured to execute instructions of a computer program. Processor 440 is therefore configured to perform at least basic arithmetic, logic, and input / output operations.

[0051] In some implementations, processor 440 may include or implement one or more engines. As used herein, the term "engine" refers to any hardware or software that is built, programmed, configured, or otherwise adapted to autonomously perform one or more functions (such as detecting electronic device 430). Processors and engines mentioned herein are, for example, any real-world device, component, or component arrangement implemented using hardware, such as by an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), or as a combination of hardware and software, such as by a microprocessor system and a set of program instructions that adapt the processor or engine to perform specific functions, which (when executed) transform the microprocessor system into a dedicated device. Processors or engines may also be implemented as a combination of both, where some functions are facilitated solely by hardware, while others are facilitated by a combination of hardware and software.

[0052] In some implementations, at least a portion (and in some cases, all) of the engine may execute on the processor of one or more computing platforms consisting of hardware that executes the operating system, system programs, and applications (e.g., one or more processors, data storage devices such as memory or drive memory, input / output facilities such as network interface devices, video devices, keyboards, mice, or touchscreen devices, etc.), while also utilizing multitasking, multithreading, distributed (e.g., cluster, peer-to-peer, cloud, etc.) processing or other such technologies where appropriate. Therefore, any processor or engine discussed herein may be implemented in a variety of physically implementable configurations and should not generally be limited to any particular implementation illustrated herein unless such limitation is explicitly stated.

[0053] In implementations, each processor or engine may consist of one or more sub-processors or sub-engines, each of which can be considered a processor or engine in itself. Furthermore, in the implementations described herein, processor 440 may correspond to defined autonomous functions, wherein the usage status of the headset 100 can be determined without additional manual input from the user. However, it should be understood that in other contemplated implementations, functions may be distributed across more than one processor or engine, regardless of any examples described or depicted herein. Similarly, in other contemplated implementations, multiple defined functions may be implemented by a single processor or engine performing these multiple functions (possibly together with other functions), or distributed across a set of processors or engines in a manner different from that specifically illustrated in the examples herein.

[0054] Therefore, the headset 100 may include any number or type of processors 440. In one embodiment, the processor 440 may be located within or locally on the headset 100. In an alternative embodiment, the processor 440 may run on a device or server remote from the headset 100, such as on an electronic device 430 (e.g., as part of an application running or presented on the electronic device 430) or in the cloud.

[0055] Memory 442 may include volatile or non-volatile memory, depending on the needs of the coupled processor 440, to provide not only space for executing instructions or algorithms, but also space for storing the instructions themselves. In embodiments, volatile memory may include, for example, random access memory (RAM), dynamic random access memory (DRAM), or static random access memory (SRAM). In embodiments, non-volatile memory may include, for example, read-only memory (ROM), flash memory, ferroelectric RAM, hard disk or optical disk memory. The foregoing list does not limit the types of memory that can be used, as these embodiments are given by way of example only and are not intended to limit the scope of this disclosure.

[0056] The power supply 448 is typically a rechargeable battery. Some implementations may use a disposable, replaceable battery. Other implementations may have the ability to receive power directly from AC or DC power sources, such as wall sockets, laptops, computers, tablets, video display devices (such as in-flight entertainment systems), or other sources.

[0057] In various embodiments, and in other embodiments, one or more of the sensors 220, IMU 444, 446 (or other sensors, components, or devices) that may be part of the headset 100 may be used to detect the usage status, position, orientation, or other characteristics of the headset 100. See also Figure 5 The detected usage status 510 or 520 of the headset 100, which uses the wearing state (on the head or not), can be used at 530 and 540 to determine, change, or customize one or more features of the headset 100. Therefore, if it is determined at 510 that the headset is on the user's head, one or more features can be enabled at 530, and vice versa at 540 and 520. In other embodiments, this can be reversed, or if wearing is detected at 510, some features can be enabled at 530 while others are disabled.

[0058] For example, if headphones 100 are detected on a user's head at point 510, the LED on headphones 100 can be turned off, and sound can be output to or from headphones 100. In another example, if headphones 100 are not detected on a user's head at point 520, the LED can be turned on and Bluetooth can be turned off. Considering the previously mentioned sensor fusion, and returning to the above... Figure 3A and 3B For relevant examples, Table 1 lists several examples.

[0059]

[0060] Table 1: Indicator Behavior and Related Sensors

[0061] The examples and others provided herein are merely some possible results of detecting usage, wearing, position / orientation, or other characteristics of the headset 100 or related to the headset. Other examples include features that may be activated or deactivated individually or in combination with any other features discussed throughout this document, including whether or which of sensors 220, 444, 446 are active; whether status LEDs or other indicators are on, off, intermittently operating (e.g., flashing), or changing color; whether the user interface is or should be active to accept user input or user input types or to provide or display output to the user; whether noise cancellation is on, off, or set in a particular manner (e.g., in transparency mode); usage or output modes (e.g., running mode, where the sound of the user's feet hitting the ground is minimized or removed by active noise cancellation (ANC); volume settings; input or output sources; or some other settings or features.

[0062] In various implementations, one or more of sensors 220, 444, and 446 may be configured to detect the usage status of the headset 100 at 510. For example, if a force on the headband 102 is sensed by sensor 220 (e.g., a force sensor) in accordance with the placement or wearing of the headset 100 on a user's head, processor 440 may be configured to activate at least one IMU sensor 444 such that IMU sensor 444 can be used to detect additional information related to the use of the headset 100, such as the specific force, rate, or orientation of the force. If IMU sensor 444 senses that the detected force is located on the headband 102 and the headband 102 extends sufficiently compared to a threshold, processor 440 may activate other corresponding sensors, such as proximity sensor 446.

[0063] In another example, where each earcup 104 of the headset 100 contains at least one IMU sensor 444, data from at least two IMU sensors 444 may be considered. In one embodiment, this may include comparing data from two IMU sensors 444 to detect when the earcups 104 are facing each other. In some implementations, the at least two IMU sensors 444 may also provide data indicating relative positioning to determine whether or to what extent the headband 102 is extended. Other embodiments may additionally utilize data from other sensors 220 (e.g., force sensors).

[0064] By using proximity sensor 446 and IMU sensor 444 in the headset 100, sensor fusion techniques can be used to increase the reliability of usage state determination, such as force orientation, force rate, force type, and surface sensation of the force (e.g., whether human skin is detected). For example, if sensor 220 detects a change in force within or on the headset 100 (e.g., when sensor 220 includes a force sensor), processor 440 can send a signal to activate IMU sensor 444 and proximity sensor 446, whereby IMU sensor 444 can be configured to detect the orientation of the force relative to the headset 100 (e.g., whether the detected change in force comes from the extension of headband 102 when the headset 100 is put on or from the retraction of headband 102 when the headset 100 is taken off, etc.), and proximity sensor 446 can be configured to detect the proximity of the headset 100 to the user (e.g., whether human skin is detected near or against ear pad 110).

[0065] Table 2 below provides a typical implementation scheme for the load and sensor values ​​for communication between the triggerable force sensor and processor 440.

[0066]

[0067] Table 2: Load and Sensor Values

[0068] As illustrated in the example above, the force sensor can be configured to detect changes in force associated with the extension or retraction (or flexing) of the headband 102. The values ​​in Table 2 indicate that the load output increases as the headband 102 extends. In some implementations, the force sensor may be sensitive enough to detect the extension or retraction of the headband 102 when a user chews or speaks while wearing the headphones 100, and to observe a small, associated force change in slight fluctuations in the LSB output. Larger force changes (such as when a user removes the headphones 100) result in larger changes in the LSB output. The signal-to-noise ratio (SNR) indicates the efficiency of each of the multiple sensors, measured as the ratio of the amplitude of the desired signal to the amplitude of the noise signal at a given point in time. A higher SNR value indicates a more sensitive force sensor to changes in the LSB.

[0069] Specific patterns of force can also be detected, such as when a user removes one earphone unit 104 from one ear while the other earphone unit 104 remains on the other ear, or when a user moves both earphone units 104 behind their ears, in front of them, or onto their neck. In some embodiments, the force sensors may detect these changes individually. In other embodiments, a combination of sensors 220 may be used to determine the use (or other) state of the headset 100 at 510, 520.

[0070] For example, wearing detection can be easier or more accurate when each of the IMU sensor 444, proximity sensor 446, and force sensor (220) is used in a sensor fusion manner. The force sensor provides sensing data related to the state of the headband 102 or the padding portion 106 (or both). The IMU sensor 444 provides sensor data related to the orientation of the headset 100, such as whether the orientation is consistent with the headset 100 being worn on the user's head and above the user's ears, or whether the headset 100 is tilted, as if the headset might be removed from the user's head and worn around the neck. The proximity sensor 446 provides sensing data about the relative position of each earpiece 104 (or one or more portions thereof) to the user's head, ears, or other body parts.

[0071] It can be seen how combining sensing data from these different sensor modalities can provide more accurate or additional information. For example, IMU sensor 444 can provide data indicating that the headset 100 is being worn around a user's neck, and data from a force sensor can confirm this if the data indicates that any force acting on the headband 102 is more consistent with neck wearing than with over-ear wearing. This can be further confirmed by considering data from proximity sensor 446 or by using both proximity sensor 446 and IMU sensor 444.

[0072] It is possible to implement various levels of complexity or precision in enabling 530 or disabling 540 one or more features. In particular, cooperation between the headset 100 and the electronic device 430 can also be considered.

[0073] Some examples involve the processing of audio output from electronic device 430 to coupled headset 100. One particular example involves a time-sensitive and (generally) difficult-to-manage task. This example is routing or routing incoming calls (which may be purely audio or audio associated with a video call) from a smartphone, which is electronic device 430, to or to said headset 100. In the example below, it is assumed that electronic device 430 and headset 100 are already coupled, for example, via Bluetooth.

[0074] refer to Figure 7 This describes an example flow 700 of call processing between electronic device 430 and headset 100, depending on the usage status of headset 100. At 702, there is no active call.

[0075] At 704, a new call is being made to electronic device 430.

[0076] At step 706, the user accepts or rejects the call. If the call is rejected, the process proceeds to step 702.

[0077] If the call is accepted, the process continues to 708, where it is determined whether the headset 100 is being worn on the user's head. This determination can be made according to any of the examples included herein, such as detecting the usage status of the headset 100 by one or more sensors 220.

[0078] If the determination at 708 indicates that the user is wearing headset 100, the process continues to 710, where headset 100 is automatically selected as the audio output device for the call. This eliminates the need for the user to choose from multiple options on how to route the call, an option that would require rapid identification and selection while the call is in progress and could result in the call being routed undesirably or even rejected.

[0079] In some implementations, it can be periodically confirmed at 712 that the headset 100 is still being worn during a call. This period can be once per second or multiple times per second, such as at least 5 times per second, at least 10 times per second, at least 20 times per second, at least 30 times per second, at least 40 times per second, at least 50 times per second, or in the range of at least 5 times per second to at least 60 times per second, or in the range of at least 10 times per second to at least 50 times per second. As long as the wearing status remains unchanged, the audio output to the headset 100 can also remain unchanged (i.e., the routine returns from 712 to 710).

[0080] If it is determined at 712 that the headset 100 is no longer being worn, the audio output can automatically return to the electronic device 430 at 714 without requiring the user to pass or return a call.

[0081] Figure 7 The process 700 can be implemented whether using headphones 100 or some other audio output device (see ibid., such as earbuds). If earbuds are used, the routine may include checking whether one or both earbuds are worn, and depending on this determination, different actions will be taken at 710. For example, if both earbuds are worn, the call can be routed at 710 as if it were for headphones 100. If only one earbud is worn, the audio output of the electronic device 430 can be routed only to that earbud, and the microphone on that earbud will also be activated for the call. In some implementations, these behaviors and options can be customized by the user, such as through applications that work in conjunction with headphones 100 or any particular audio output device.

[0082] Additional applications of these concepts can be used. For example, Figure 7The process 700 can be applied to determine whether to route the audio of a call between a mobile phone, a vehicle, or a headset. An incoming call may initially be routed to the vehicle's speaker. Then, if a headset is detected being worn at 708, the call audio can be passed to the headset. In this example, additional determination of whether the vehicle is active may be included. Furthermore, reverse routing can be implemented, such as passing the call audio from the headset to the vehicle when a user enters the vehicle and removes their headset.

[0083] Therefore, these and other examples may include:

[0084]

[0085] Table 3: Example Features and Corresponding Sensors

[0086] As previously mentioned, these and other features may also be combined based on sensed wearing status or other activity. In one implementation, these features are pre-programmed and user-selectable. In other implementations, the user can customize which features(s) are implemented in different wearing statuses or sensed use.

[0087] These selections and customizations can be made on an application (“app”), a web-based application, or any other executable application framework running on an electronic device 430 communicatively coupled to the headset 100, such as a smartphone, smartwatch, or other wearable device (such as a fitness watch or tracker), tablet, e-reader, laptop, computer, or other computing device capable of interacting with the headset 100 (e.g., via Bluetooth) and hosting or presenting the app to a user. The term “electronic device” is used throughout this document for convenience but does not limit the actual characteristics, features, or composition of the device or any device that embodies the electronic device 430. The headset 100 is generally configured to provide bidirectional data communication with the electronic device 430 via a wired or wireless connection. In alternative embodiments, an electronic device having earphone units (e.g., earbuds) is configured to provide bidirectional data communication with the electronic device 430 via a wired or wireless connection.

[0088] The user interface can be configured to receive user input and provide output regarding the configuration and status of the headset 100, or alternatively, any electronic device characterized as a headphone unit. The user interface can allow for personalized system control and calibration, such as enabling the user to calibrate one or more active usage states by placing the headset in the desired position and recording sample force data. In an embodiment, the electronic device 430 may be associated with one or more user profiles, each of which may represent different feature processing requirements based on the detected usage state of the headset 100 or any electronic device including the headphone unit.

[0089] Processor 440 can automatically configure one or more features of the headset 100 based at least in part on the received sensor data. By processing sensing data relating to the force exerted from the headband 102 onto the earcups 108 (from the force sensor of sensor 220) and the orientation of the force (from IMU sensor 444 in one example), as well as detected capacitance or proximity (from proximity sensor 446), processor 440 can determine whether the headset 100 is on the user's head and thus can represent an active usage instance. In an alternative embodiment, processor 440 can process the detected sensor data from the force sensor of sensor 220, IMU sensor 444, and proximity sensor 446 to determine whether the in-ear earpieces are in the user's ears and thus determine whether it is in an active usage state. In these and other examples, other embodiments may use sensor modes other than capacitance and force. For example, some embodiments may include infrared (IR) sensors, pulse sensors, ultrasonic sensors, or other sensor modes, in lieu of or supplementing any sensor modes discussed herein with respect to sensor 220.

[0090] Specifically, the relative increase in force when the headband 102 extends (e.g., worn around or above the user's head) can be implemented within the processor 440 to identify active or inactive usage states and control the characteristics of the headset 100 accordingly. Active usage states may include any situation where the user might expect the headset to produce sound, provide noise cancellation, or generate other outputs. Inactive usage states may include one or more of the following: placing the headset 100 on a flat surface, folding it, storing it in a case, wearing it around the neck, placing it on a headset stand, or holding it by the user.

[0091] In an alternative implementation, processor 440 may identify differences in the forces applied to the outer surface of the electronic device, including the headphone unit (e.g., in-ear earbuds rather than over-ear headphones), to determine whether the usage state is active or inactive. As indicated above, an active usage state may include any situation in which the user might expect the electronic device to produce sound or provide noise cancellation functionality. An inactive usage state may include one or more of the following: placing the electronic device on a flat surface, folding it, storing it in a box or container, placing it on a stand, or holding it by the user.

[0092] The embodiments of this disclosure may optionally implement artificial intelligence (AI) or machine learning (ML) to better identify patterns or changes in sensor data related to wearing, wearing, removing, tilting, and other processing of the headset 100, headphones, or other electronic devices. Patterns or changes in forces, etc., sensed on the headset 100 during changes in use or wearing status can be extracted manually or automatically using machine learning methods such as convolutional neural networks to generate training data that can be compared with data received during use. Thus, the ML model can be applied to labeled (supervised) sensor data (data representing known transitions in use status) by the processor 440. In the embodiments, unlabeled (unsupervised) sensor data can be used, although the accuracy and precision of the ML model will be relatively poor without additional training.

[0093] Through training, the ML model can better identify when differences in sensor data may indicate a change in usage state. In an implementation, the comparison process can be performed by calculating a similarity metric using a correlation or machine learning regression algorithm. For example, if the similarity of the "remove headphones" gesture is higher than a certain threshold (e.g., 75%, 90%, 95%, or 99% similarity), the matching process can determine that the received sensor data indicates a change in usage state, and the characteristics of the headphones 100 can be controlled accordingly, such as turning off the headphones 100.

[0094] This analysis can be improved by incorporating a feedback loop for classifying sensor data patterns for a specific user. Alternative implementations may include analyzing gestures related to headphones 100, earphones, or other electronic devices. For example, a "remove earbuds" gesture could exceed a certain threshold (as discussed in the previous example regarding headphones 100). As more comparisons of the received data and training data are completed, feedback on the accuracy of previous comparisons can be tracked to better identify future sensor data patterns.

[0095] In the implementation, the processor 440 may also implement one or more classifiers to consider parameters such as the type of electronic device (e.g., headset 100 versus headphones versus VR goggles) and the types of force sensor 220, IMU sensor 444, and proximity sensor 446.

[0096] During operation, a force applied to the outside of one or both earcups 108 by the headband 102 (or the user's head or ears) can automatically trigger a signal to power on the IMU sensor 444 and the proximity sensor 446. A force sensor 220, which may be located within one or both earcups 108 of the headset 100 or on or inside the earbuds, can work in conjunction with the IMU sensor 444 and the proximity sensor 446 to detect changes in force from the headband 102 to the earcups 108, wherein the detected force can be positive (e.g., expanding or extending the headband 102 / moving the earcups 108 away from each other) and negative (relaxing or retracting the headband 102 / moving the earcups 108 closer together). The detected change in force can automatically trigger an electronic device, such as headphones 100, to power on automatically, and can also automatically trigger control of features based on active usage status. These features may include (in addition to those mentioned herein) establishing or disconnecting a Bluetooth connection between the electronic device, such as headphones 100, and electronic device 430, call routing, enabling or disabling user input mechanisms (such as touch-based user interfaces) for the electronic device, such as headphones 100, resuming or pausing media playback, changing playback volume, and switching between active noise cancellation level mode and transparency mode.

[0097] In the implementation, the AI ​​or ML system may be included in or communicatively coupled to processor 440 to further enable the headset 100 or electronic device to respond to one or more settings or features based on a combination of data received from sensors (including sensors 220, 444, 446) and data received from one or more other sources (including electronic device 430 or other sources). This data may include location data, calendar data, proprioceptive data, health and wellness data, or other relevant user data. For example, one or more profiles may be created in response to data processing by the AI ​​system to reflect typical user behavior, activities, or preferred settings.

[0098] Therefore, embodiments of this disclosure provide the detection and subsequent control of the usage status of one or both of electronic devices and headphones or in-ear earphones, such as for improving or automatically routing call audio and providing status information via indicators such as LEDs. By combining sensor modalities and data (e.g., wear detection and force) and efficiently processing this data, the use of these devices and the routing of audio signals can be made more convenient and intuitive.

[0099] It should be understood that the examples discussed in this article can also be applied to situations using multiple audio output devices. For example, a home theater system may have multiple sets of headphones for users to wear, allowing them to enjoy audio from a music system (stereo or streaming source) and a television simultaneously. Audio signals can only be routed to those devices that are determined to be active or worn.

[0100] The following terms are part of this disclosure.

[0101] Clause 1: A system for improving the routing of audio output signals from an electronic device to an audio output device, the system comprising: at least one inertial measurement unit (IMU) disposed in the audio output device; and at least one processor configured to: receive data from the at least one IMU; determine from the received data whether the current state of the audio output device is worn or not; if the current state of the audio output device is worn, route the audio output signal from the electronic device to the audio output device; and if the current state of the audio output device is not worn, route the audio output signal to the audio output of the electronic device.

[0102] Clause 2: The system as described in Clause 1, wherein the audio output device comprises: an earbud having an IMU; a set of two earbuds, wherein a first earbud of the two earbuds has a first IMU disposed therein, and a second earbud of the two earbuds has a second IMU disposed therein; or a set of headphones, wherein a first earphone unit of the set of headphones has a first IMU disposed therein, and a second earphone unit of the set of headphones has a second IMU disposed therein.

[0103] Clause 3: A system as described in Clause 1 or Clause 2, wherein the electronic device is a smartphone, tablet computer, e-reader, wearable device, smartwatch, laptop computer, gaming device, computer, stereo system, tuner, smart home device, home automation device, camera, television, home appliance, fitness equipment, or vehicle.

[0104] Clause 4: The system as described in any one of Clauses 1-3, wherein the current state of the audio output device being worn includes partial wearing.

[0105] Clause 5: The system of any one of Clauses 1-4, wherein the at least one processor is further configured to, after routing the audio output signal from the electronic device to the audio output device if the current state of the audio output device is worn: periodically re-determines the current state of the audio output device as worn or not worn from received data; if the re-determined current state of the audio output device is worn, continues to route the audio output signal from the electronic device to the audio output device; and if the re-determined current state of the audio output device is not worn, stops routing the audio output signal to the audio output device and begins to route the audio output signal to the audio output of the electronic device.

[0106] Clause 6: A system as described in any one of Clauses 1-5, wherein the system further includes a status indicator of the audio output device, and the at least one processor is further configured to change the status of the status indicator based on received data.

[0107] Clause 7: An audio output device comprising the system described in any one of Clauses 1-6.

[0108] Clause 8: A method for routing an audio output signal from an electronic device to an audio output device, the method comprising: acquiring data from at least one inertial measurement unit (IMU) of the audio output device; determining from the acquired data whether the current state of the audio output device is worn or not; if the current state of the audio output device is worn, routing the audio output signal from the electronic device to the audio output device; and if the current state of the audio output device is not worn, routing the audio output signal to an audio output of the electronic device.

[0109] Clause 9: The method as described in Clause 8, wherein the audio output device comprises: an earbud having an IMU; a set of two earbuds, wherein a first earbud of the two earbuds has a first IMU disposed therein, and a second earbud of the two earbuds has a second IMU disposed therein; or a set of headphones, wherein a first earphone unit of the set of headphones has a first IMU disposed therein, and a second earphone unit of the set of headphones has a second IMU disposed therein.

[0110] Clause 10: As described in Clause 8 or Clause 9, wherein the electronic device is a smartphone, tablet computer, e-reader, wearable device, smartwatch, laptop computer, gaming device, computer, stereo system, tuner, smart home device, home automation device, camera, television, home appliance, fitness equipment, or vehicle.

[0111] Clause 11: The method of any one of Clauses 8-10, wherein the current state of the audio output device being worn includes partial wearing.

[0112] Clause 12: The method of any one of Clauses 8-11, the method further comprising: if the current state of the audio output device is worn, then after routing the audio output signal from the electronic device to the audio output device: periodically re-determining from acquired data whether the current state of the audio output device is worn or not; if the re-determined current state of the audio output device is worn, then continuing to route the audio output signal from the electronic device to the audio output device; and if the re-determined current state of the audio output device is not worn, then stopping the routing of the audio output signal to the audio output device and starting to route the audio output signal to the audio output of the electronic device.

[0113] Clause 13: The method of any one of Clauses 8-12, the method further comprising changing the state of a status indicator of the audio output device based on the acquired data.

[0114] Various embodiments of the systems, apparatus, and methods have been described herein. These embodiments are given by way of example only and are not intended to limit the scope of the claimed invention. Furthermore, it should be understood that the various features of the described embodiments can be combined in various ways to produce many additional embodiments. In addition, although various materials, sizes, shapes, configurations, and locations, etc., for the disclosed embodiments have been described, others besides those disclosed may also be used without exceeding the scope of the claimed invention.

[0115] Those skilled in the art will recognize that the subject matter of this disclosure may include fewer features than those shown in any single embodiment described above. The embodiments described herein are not intended to exhaustively represent all possible combinations of features of the subject matter of this disclosure. Therefore, the embodiments are not mutually exclusive combinations of features; rather, as will be understood by those skilled in the art, various embodiments may include combinations of different individual features selected from different individual embodiments. Furthermore, unless otherwise stated, elements described with respect to one embodiment may be implemented in other embodiments, even if not described in such embodiments.

[0116] While a dependent claim may refer in the claim to a specific combination with one or more other claims, other embodiments may also include a combination of the subject matter of that dependent claim with each of the other dependent claims, or a combination of one or more features with other dependent or independent claims. Such combinations are presented herein unless stated otherwise.

[0117] Any inclusion of the foregoing documents by reference is limited to ensuring that no subject matter contradicting the express disclosure herein is incorporated. Any inclusion of the foregoing documents by reference is further limited to ensuring that no claims included in the documents are incorporated herein by reference. Any inclusion of the foregoing documents by reference is also further limited to ensuring that any definitions provided in the documents are not incorporated herein by reference unless expressly included herein.

[0118] To interpret the claims, the explicit intent is to apply a means-plus-function interpretation, unless the specific terms "means for..." or "steps for..." are recorded in the claims.

Claims

1. A system for improving the routing of audio output signals from an electronic device to an audio output device, the system comprising: At least one inertial measurement unit (IMU) is arranged in the audio output device; as well as At least one processor, said at least one processor being configured to: Receive data from the at least one IMU. The current state of the audio output device is determined from the received data as to whether it is being worn or not. If the current state of the audio output device is "worn", then the audio output signal is routed from the electronic device to the audio output device, and If the current state of the audio output device is not being worn, the audio output signal is routed to the audio output of the electronic device.

2. The system of claim 1, wherein the audio output device comprises: Earplugs, the earplugs having an IMU; or A set of two earbuds, wherein a first IMU is disposed in the first earbud of the two earbuds, and a second IMU is disposed in the second earbud of the two earbuds; or A set of over-ear headphones, wherein a first IMU is arranged in a first earphone unit of the set of over-ear headphones, and a second IMU is arranged in a second earphone unit of the set of over-ear headphones.

3. The system of claim 1, wherein the electronic device is a smartphone, tablet computer, e-reader, wearable device, smartwatch, laptop computer, gaming device, computer, stereo system, tuner, smart home device, home automation device, camera, television, home appliance, fitness equipment, or vehicle.

4. The system of claim 1, wherein the current state of the audio output device being worn includes partial wearing.

5. The system of claim 1, wherein the at least one processor is further configured to route the audio output signal from the electronic device to the audio output device if the current state of the audio output device is worn: The current state of the audio output device, whether it is worn or not, is periodically re-determined based on the received data. If the current state of the re-determined audio output device is that it is being worn, then the audio output signal continues to be routed from the electronic device to the audio output device, and If the current state of the redefined audio output device is not being worn, then routing the audio output signal to the audio output device is stopped, and routing the audio output signal to the audio output of the electronic device begins.

6. The system of claim 1, wherein the system further includes a status indicator of the audio output device, and the at least one processor is further configured to change the status of the status indicator based on received data.

7. A method for routing an audio output signal from an electronic device to an audio output device, the method comprising: Data is acquired from at least one inertial measurement unit (IMU) of the audio output device; Based on the acquired data, determine whether the current state of the audio output device is being worn or not. If the current state of the audio output device is "worn", then the audio output signal is routed from the electronic device to the audio output device; as well as If the current state of the audio output device is not being worn, the audio output signal is routed to the audio output of the electronic device.

8. The method of claim 7, wherein the audio output device comprises: Earplugs, the earplugs having an IMU; or A set of two earbuds, wherein a first IMU is disposed in the first earbud of the two earbuds, and a second IMU is disposed in the second earbud of the two earbuds; or A set of over-ear headphones, wherein a first IMU is arranged in a first earphone unit of the set of over-ear headphones, and a second IMU is arranged in a second earphone unit of the set of over-ear headphones.

9. The method of claim 7, wherein the electronic device is a smartphone, tablet computer, e-reader, wearable device, smartwatch, laptop computer, gaming device, computer, stereo system, tuner, smart home device, home automation device, camera, television, home appliance, fitness equipment, or vehicle.

10. The method of claim 7, wherein the current state of the audio output device being worn includes partial wearing.

11. The method of claim 7, further comprising, if the current state of the audio output device is worn, routing the audio output signal from the electronic device to after the audio output device: The current state of the audio output device is periodically re-determined as whether it is being worn or not based on the acquired data. If the current state of the re-determined audio output device is that it is being worn, then the audio output signal continues to be routed from the electronic device to the audio output device, and If the current state of the redefined audio output device is not being worn, then routing the audio output signal to the audio output device is stopped, and routing the audio output signal to the audio output of the electronic device begins.

12. The method of claim 7, further comprising changing the state of a status indicator of the audio output device based on the acquired data.

Images