System and method for configuring device features based on detected device status

Abstract

The present disclosure includes systems and methods for improving robustness of electronic device usage state detection. In aspects, the current usage state of the electronic device is compared to a previous usage state. And if the current use state is different from the previous use state, changing the state of at least one feature of the electronic device based on the current use state. If the current usage state is the same as the previous usage state, a state of at least one feature of the electronic device may be maintained.

Description

Technical Field

[0001] The embodiments disclosed herein relate generally to consumer electronic devices, and more specifically to systems and methods for configuring features of consumer electronic devices based on detected device states. Background Technology

[0002] Electronic device management is typically accomplished by users manually controlling the device's power and features through inputs such as buttons, dial pads, and touch surfaces or touchscreens. This manual control of device features can be cumbersome and detract from the overall user experience when interacting with electronic devices.

[0003] Typically, users often need to adjust or otherwise control features before, during, and after each use. This issue particularly affects wireless headphones, as headphones (and often earbuds) require a series of manual user steps to configure before, during, and after each activity.

[0004] There are known methods for automatically enabling or disabling electronic devices, or for turning electronic devices on or off. These methods are often basic, requiring the use of multiple devices (e.g., interacting with a smartphone or other device as the source of content delivered to, for example, headphones or earbuds), or are prone to errors. Common errors are false alarms or false negatives in usage status detection, such as disabling audio output to a pair of headphones when a smartphone incorrectly determines that the headphones are not in use. While some errors may be insignificant or only slightly annoying, others can be problematic or destructive, such as incorrectly detecting lack of use and muting the sound during an important call or meeting, or incorrectly disabling user interface features when the user needs to quickly change settings (e.g., volume).

[0005] Therefore, these shortcomings still need to be addressed. Summary of the Invention

[0006] Various embodiments of this disclosure aim to address the aforementioned problems, including by making electronic devices more robust in use and handling, making them less prone to false alarms and missed alarms in usage status detection and other common errors. Therefore, this disclosure relates to systems and methods for improving the robustness of usage status detection in electronic devices.

[0007] In one embodiment, a system for improving the robustness of detecting the usage status of an electronic device includes a plurality of sensors, including at least one first sensor arranged to acquire first type of data about the electronic device, and at least one second sensor arranged to acquire second type of data about the electronic device, the second type of data being different from the first type of data. The system also includes at least one processor coupled to receive the first type of data from the at least one first sensor and the second type of data from the at least one second sensor, and from the first type of data and the second type of data: determine the current usage status of the electronic device; compare the current usage status of the electronic device with a previous usage status; if the current usage status is different from the previous usage status, change the status of at least one feature of the electronic device based on the current usage status; and if the current usage status is the same as the previous usage status, maintain the status of at least one feature of the electronic device.

[0008] In another embodiment, a method for improving the robustness of detecting the usage status of an electronic device includes acquiring a first type of data about the electronic device; acquiring a second type of data about the electronic device, the second type of data being different from the first type of data; determining the current usage status of the electronic device from the first type of data and the second type of data; comparing the current usage status of the electronic device with a previous usage status; if the current usage status is different from the previous usage status, changing the status of at least one feature of the electronic device based on the current usage status; and if the current usage status is the same as the previous usage status, maintaining the status of at least one feature of the electronic device.

[0009] The foregoing overview is not intended to describe every embodiment or implementation of the subject matter of this disclosure. The accompanying figures and detailed description below illustrate various embodiments in more detail. Attached Figure Description

[0010] The subject matter of this disclosure can be more fully understood by considering the following detailed descriptions of various embodiments in conjunction with the accompanying drawings, wherein:

[0011] Figure 1 This is a three-dimensional view of the headset according to the implementation plan.

[0012] Figure 2A This is a perspective view of the headphone unit of a headset according to the implementation plan.

[0013] Figure 2B yes Figure 2A A partial 3D view of the headphone unit.

[0014] Figure 3It is a block diagram of the system based on the implementation plan.

[0015] Figure 4 It is based on the functional diagram of the implementation plan.

[0016] Figure 5 It is a flowchart of the method according to the implementation plan.

[0017] Figure 6A and Figure 6B This is a flowchart of a method for detecting usage status according to the implementation plan.

[0018] While various embodiments may have various modifications and alternatives, their details have been shown by way of illustration in the figures and will be described in detail. However, it should be understood that the claims are not intended to be limited to the specific embodiments described. Rather, they are intended to cover all modifications, equivalents, and alternatives that fall within the spirit and scope of the subject matter defined by the claims. Detailed Implementation

[0019] Embodiments of this disclosure relate to making electronic devices more robust in handling and use. This may include, in some embodiments, reducing or eliminating false alarms and false negatives in usage status detection. Thus, embodiments may provide more accurate automatic activation, configuration, or deactivation of one or more features of the device based on detected usage status or usage conditions.

[0020] As used herein, “usage state” refers to the operational state of a device relative to its user, such as a pair of headphones. Usage state can include wearing state, i.e., whether a pair of headphones is being worn or is being worn as intended (e.g., where each earcup is worn above the ear and the headband extending between the earcups rests on the user's head). Thus, usage state can be on and not worn, or on and partially worn. Usage state can also include operational state (e.g., on or off) or setting state, such as whether active noise cancellation (ANC) is on or which ANC mode or setting is selected. Usage state is generally used in a broad sense herein (including wearing state, operational state, and setting state, or at least several of these), although wearing state or operational state may be specifically mentioned or discussed in some instances or examples.

[0021] refer to Figure 1 A pair of headphones 100 is depicted according to an embodiment. The headphones 100 include a headband 102 that pivotally or flexibly couples headphone units 104 to both ends. The headband 102 is adjustable such that the length of the headband 102 between the headphone units 104 can be shortened, lengthened, or otherwise adjusted to better or more comfortably fit a particular user. The headband 102 may optionally include a padded portion 106 to further enhance user comfort when wearing the headphones.

[0022] Each headphone unit 104 includes an earcup 108 and an ear pad 110. The earcup 108 houses electronic components, such as speaker drivers and associated circuitry, which are configured to generate sound when the headphones 100 are worn and to project sound into the user's ears within the ear pad 110.

[0023] One or more outer surfaces or portions of each earcup 108 may include at least one input 112 for receiving user input. Input 112 may be one or more of a button, slider, touch-sensitive surface, etc. Input 112 may be configured to receive user input to control, for example, the power, volume, noise cancellation, fit, comfort, noise isolation, and other features of the headset 100. Input 112 may be located anywhere on the headset 100 such that it is available for manual input by the user when worn or as needed.

[0024] In some embodiments, the earcups 108 may further include one or more input ports 114 configured to receive a cable connector and one or more indicator lights 116 configured to convey status information of the headset 100. These input ports 114 may include audio ports or jacks, such as USB or mini-jacks as examples. The arrangement of the inputs 112, input ports 114, and indicator lights 116 may vary between the earcups 108 (e.g., left and right) or in different embodiments or versions of the headset 100.

[0025] although Figure 1 The implementation of the over-ear headphones 100 (and other implementations discussed herein) generally relates to or discusses devices worn above a user's ears, but this disclosure and various implementations are not limited to over-ear headphones. Therefore, aspects of this disclosure may also relate to in-ear headphones (often referred to as "earplugs"), headphones worn on but not completely covering the ears, one or more headphone units or components disposed within other devices (such as headsets or wearable devices such as hats, glasses, or headbands) that position the headphones relative to one or more of a user's ears, and other audio, video, or audio / video (A / V) devices, unless otherwise expressly stated herein.

[0026] refer to Figures 2A to 2B According to an implementation plan Figure 2A The internal portion of the headphone unit 104 coupled to the headband 102 is depicted. Figure 2B yes Figure 2AA rotating close-up perspective view of region 212 of the headphone unit 104. The headphone unit 104 includes an earcup 108 configured to contain a speaker driver (not shown), at least one sensor 220, and processing hardware (not shown). As previously described, the headphone unit 104 is coupled to the headband 102 via a coupling mechanism 222, which in this embodiment includes a hinge 224. Other coupling mechanisms may be used in other embodiments.

[0027] In the implementation, at least one sensor 220 includes a force sensor (see...). Figure 3 (320 in the original text) The force sensor is configured to detect at least one characteristic associated with the headphones 100, such as the force exerted by the headband 102 or another object on one or both of the earphone units 104. For example, when the headphones 100 are worn on a user's head, the headband 102 is biased to flex or bend outward to exert a generally inward force (i.e., toward the user's head or ears) on each earphone unit 104. This bias provides a good "seal" of the ear pads 110 around each of the user's ears for a better audio experience (e.g., improved noise cancellation), but not so strong as to affect a comfortable fit.

[0028] Sensor 220 is communicatively coupled to onboard processing hardware, which may include at least one processor and memory on which firmware / software may reside. In some embodiments, data collected by sensor 220 may be communicated alternatively or additionally to a server or user equipment (e.g., a smartphone) communicatively coupled to or remote from headset 100.

[0029] In some implementations, one or more additional sensor modes may be used in conjunction with or in place of a force sensor, such as an inertial measurement unit (IMU) sensor for sensing motion orientation, an accelerometer for sensing acceleration, a gyroscope for sensing orientation, a magnetometer or Hall effect sensor for sensing magnetic fields, a proximity sensor, a capacitive touch sensor for sensing touch, a millimeter (MM) wave sensor for sensing reflected signals indicating the angle, range, and velocity of a sensed object, an infrared sensor for sensing thermal radiation, a temperature sensor for sensing temperature, a humidity sensor for sensing humidity, or one or more other sensors known to those skilled in the art. In implementations that implement multiple sensor modes, so-called “sensor fusion” may be achieved by or for headset 100. Sensor fusion combines data from multiple sensors or sources, reducing uncertainty in data processing or providing additional information compared to what a single sensor mode might provide individually.

[0030] Therefore, a first sensor 220 of a first sensor mode may be arranged in or on the headset 100 to obtain a first type of data about the headset 100, wherein the first type of data is data of the first sensor mode (e.g., force sensor data, IMU sensor data, temperature sensor data). A second sensor 220 of a second sensor mode may be arranged in or on the headset 100 to obtain a second type of data about the headset 100, wherein the second type of data is data of the second sensor mode (e.g., IMU sensor data, accelerometer data, Hall sensor data). The first sensor mode may be different from the second sensor mode, and multiple first sensors 220 or second sensors 220 or both may be present. Sensor fusion may be applied to some or all of the first and second types of data (e.g., sensor data). In some embodiments, machine learning or artificial intelligence techniques may be applied to some or all of the data or the data of the aforementioned types (as will be discussed in more detail below). In other embodiments, a third or more (fourth, fifth, etc.) sensor modes may also be used to obtain a third type of data, etc. The first, second, and third types of data can all be different (e.g., force sensor data, IMU sensor data, capacitive sensor data), or some can be the same. In any embodiment related to the headset 100 or, similarly, headphones, the sensor mode in one headphone unit 104 may be the same as or different from the sensor mode in another headphone unit 104, or sensor 220 may be omitted from one or both headphone units. Sensor 220 may also be included in or on the headband 102 or elsewhere in or on the headset 100.

[0031] refer to Figure 3 A block diagram of a system 300 for detecting the usage status of a headset 100 is depicted according to an embodiment. The system 300 includes a headset 100 and a user equipment 330. The headset 100 (which can be connected to...) Figure 1-2B The headset 100 (as in the example) generally includes a processor 340, memory 342, one or more force sensors 320, one or more IMU sensors 344, one or more proximity sensors 346, one or more sensors of optional other sensor modes, and at least one power supply 348.

[0032] System 300 can be used to determine the usage state of the headset 100 (e.g., worn for active use vs. not worn vs. partially worn), which can be used to configure one or more features or operations of the headset 100. As discussed elsewhere herein, the features and concepts of this disclosure mentioned in examples related to the headset 100 can be used in or applied to other devices, such as in-ear headphones and other electronic devices with user interface features, for which determining the usage state and configuring or controlling one or more features or operations based on the determined usage state may be desirable and helpful. Therefore, in various embodiments, the headset 100 may alternatively be any electronic device incorporating one or more headphone units, the electronic device comprising at least in-ear earbuds or on-ear headphones. The headset 100 may alternatively include various wearable electronic devices, such as virtual reality (VR) headsets, augmented reality (AR) headsets, smartwatches, smart glasses, smart jewelry or other smart accessories, gaming devices, medical or health devices, or some other electronic devices worn or used on or near a user's body or body part.

[0033] Processor 340 (and any other processor, processing device, or engine discussed herein) can be any programmable device that accepts digital data as input, is configured to process the input according to instructions or algorithms, and provides results as output. In one embodiment, processor 340 can be a central processing unit (CPU) or a microcontroller or microprocessor configured to execute instructions of a computer program. Processor 340 is therefore configured to perform at least basic arithmetic, logic, and input / output operations.

[0034] In some implementations, processor 340 may include or implement one or more engines. As used herein, the term "engine" refers to any hardware or software that is built, programmed, configured, or otherwise adapted to autonomously perform one or more functions, such as detecting user equipment 330. Processors and engines mentioned herein are, for example, any real-world device, component, or component arrangement implemented using hardware, such as by an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), or as a combination of hardware and software, such as by a microprocessor system and a set of program instructions that adapt the processor or engine to perform specific functions, which (when executed) transform the microprocessor system into a dedicated device. Processors or engines may also be implemented as a combination of both, where some functions are facilitated solely by hardware, while others are facilitated by a combination of hardware and software.

[0035] In some implementations, at least a portion (and in some cases, all) of the engine may execute on the processor of one or more computing platforms consisting of hardware that executes the operating system, system programs, and applications (e.g., one or more processors, data storage devices such as memory or drive memory, input / output facilities such as network interface devices, video devices, keyboards, mice, or touchscreen devices, etc.), while also utilizing multitasking, multithreading, distributed (e.g., cluster, peer-to-peer, cloud, etc.) processing or other such technologies where appropriate. Therefore, any processor or engine discussed herein may be implemented in a variety of physically implementable configurations and should not generally be limited to any particular implementation illustrated herein unless such limitation is explicitly stated.

[0036] In implementations, each processor or engine may consist of one or more sub-processors or sub-engines, each of which can be considered a processor or engine in itself. Furthermore, in the implementations described herein, processor 340 may correspond to defined autonomous functions, wherein the usage status of the headset 100 can be determined without additional manual input from the user. However, it should be understood that in other contemplated implementations, functions may be distributed across more than one processor or engine, regardless of any examples described or depicted herein. Similarly, in other contemplated implementations, multiple defined functions may be implemented by a single processor or engine performing these multiple functions (possibly together with other functions), or distributed across a set of processors or engines in a manner different from that specifically illustrated in the examples herein.

[0037] Therefore, the headset 100 may include any number or type of processors 340. In one embodiment, the processors 340 may be located within or locally on the headset 100. In alternative embodiments, the processors 340 may run on a device or server remote from the headset 100, such as on a user device 330 (e.g., as part of an application running or presented on the user device 330) or in the cloud.

[0038] Memory 342 may include volatile or non-volatile memory, depending on the needs of the coupled processor 340, to provide not only space for executing instructions or algorithms, but also space for storing the instructions themselves. In embodiments, volatile memory may include, for example, random access memory (RAM), dynamic random access memory (DRAM), or static random access memory (SRAM). In embodiments, non-volatile memory may include, for example, read-only memory (ROM), flash memory, ferroelectric RAM, hard disk drive, or optical disk drive. The foregoing list is by no means limiting the types of memory that can be used, as these embodiments are given by way of example only and are not intended to limit the scope of this disclosure.

[0039] Power source 348 is typically a rechargeable battery. Some implementations may use a disposable, replaceable battery. Other implementations may have the ability to receive power directly from AC or DC power sources, such as wall sockets, laptops, computers, tablets, video display devices (such as in-flight entertainment systems), or other sources.

[0040] Continuously operating all components of the headset 100 consumes a significant amount of battery life, making it useful to use one or more of the sensors 220 to detect the usage status of the headset 100 in order to determine which features receive active power and which features (including the entire headset 100) can be disabled because they are inactive or unnecessary. Furthermore, detecting the usage status of the headset 100 is also useful in determining whether user input to (or output from) the headset 100 is expected or unexpected.

[0041] Therefore, in various embodiments, and in other embodiments, one or more of the sensors 220 (or other sensors, components, or devices) used with the headphones 100 may be used to detect the usage status of the headphones 100. See also Figure 4 The detected usage status 410 or 420 of the headset 100 can be used at 430 and 440 to determine, change, or customize one or more features of the headset 100. Figure 4 In this implementation, if it is determined at 410 that the headset is on the user's head, one or more features are depicted as enabled at 430, and vice versa at 420 and 440. In other implementations, this can be reversed, or if wearing is detected at 410, some features can be enabled at 430 while others are disabled simultaneously. For example, if it is detected at 410 that the headset 100 is on the user's head, the LED on the headset 100 can be turned off, and sound can be output to or from the headset 100. In another example, if it is not detected at 420 that the headset 100 is being worn on the user's head, the LED can be turned on and Bluetooth can be turned off.

[0042] These examples are just some possible outcomes of detecting whether the headset 100 is being worn. Other examples of features that may be activated or deactivated individually or in combination with any other features discussed throughout this document include whether or which of the sensors 220 are active; whether status LEDs or other indicators are on, off, intermittently running (e.g., flashing), or changing color; whether the user interface is or should be active to accept user input or user input types or to provide or display output to the user; whether noise cancellation is on, off, or set in a particular way (e.g., in transparency mode); usage or output modes (e.g., running mode, where the sound of the user's feet hitting the ground is minimized or removed by active noise cancellation (ANC); volume settings; input or output sources; or some other settings or features. Additional examples will be provided below because various levels of complexity or precision can be achieved by enabling 430 or disabling 440 one or more features based on various factors that will be explained further.

[0043] In various implementations, one or more of the sensors 220 may be configured to detect the usage status of the headset 100 at 410. For example, if the force sensor 320 senses a force on the headband 102 consistent with the headset 100 being placed or worn on the user's head, the processor 340 may be configured to activate at least one IMU sensor 344 such that the IMU sensor 344 can be used to detect additional information related to the use of the headset 100, such as a specific force, rate, or orientation of force sensed by the force sensor 320. If the IMU sensor 344 senses that the detected force is located on the headband 102 and the headband 102 extends sufficiently compared to a threshold, the processor 340 may activate other corresponding sensors, such as a proximity sensor 346.

[0044] In another example, where each earphone unit 104 of the headset 100 has at least one IMU sensor 344, data from at least two IMU sensors 344 may be considered. In one embodiment, this may include comparing data from two IMU sensors 344 to detect when the earphone units 104 are facing each other. In some implementations, the at least two IMU sensors 344 may also provide data indicating relative positioning to determine whether or to what extent the headband 102 is extended. Other embodiments may additionally utilize data from a force sensor 320.

[0045] In one example, force sensor 320 can be configured to remain continuously active, even when other parts of the headset 100 are powered off or switched off. Advantageously, force sensor 320 uses very little power, particularly compared to IMU sensor 344 and proximity sensor 346, and therefore this configuration allows the user to obtain optimal use without significantly depleting the battery life of the headset 100. In an alternative example, force sensor 320 can be configured to be powered off along with the headset 100, or other sensors can be configured to remain on (such as IMU sensor 344, proximity sensor 346, etc.). By configuring one or more of the sensors 220 to remain active, the lifespan of power supply 348 can be managed and conserved while maintaining the ability to detect any user interaction with the headset 100 and subsequently wake up the entire system for further detection and evaluation of the detected force (e.g., activity).

[0046] By using proximity sensor 346 and IMU sensor 344 in the headset 100, sensor fusion technology can be used to obtain greater confidence in the determination of usage status, such as the orientation of force, the rate of force, the type of force, and the surface sensation of force (e.g., whether human skin is detected). For example, if force sensor 320 detects a change in force within or on the headset 100, processor 340 can send a signal to activate IMU sensor 344 and proximity sensor 346, whereby IMU sensor 344 can be configured to detect the orientation of the force relative to the headset 100 (e.g., whether the detected change in force comes from the extension of headband 102 when the headset 100 is put on or from the retraction of headband 102 when the headset 100 is taken off, etc.), and proximity sensor 346 can be configured to detect the proximity of the headset 100 to the user (e.g., whether human skin is detected approaching or against ear pad 110).

[0047] Table 1 below provides a typical implementation scheme for the load and sensor values ​​for communication between the triggerable force sensor 320 and the processor 340.

[0048]

[0049] Table 1: Load and Sensor Values

[0050] As illustrated in the example above, force sensor 320 can be configured to detect changes in force associated with the extension or retraction (or flexing) of headband 102. The values ​​in Table 1 indicate that the load output increases as headband 102 extends. In some implementations, force sensor 320 may be sensitive enough to detect the extension or retraction of headband 102 when a user chews or speaks while wearing headphones 100, and to observe a small, associated force change in slight fluctuations in LSB output. Larger force changes (such as when a user removes headphones 100) result in larger changes in LSB output. Signal-to-noise ratio (SNR) indicates the efficiency of each of the multiple sensors, measured as the ratio of the amplitude of the desired signal to the amplitude of the noise signal at a given point in time. The higher the SNR value, the more sensitive force sensor 320 is to changes in LSB.

[0051] Specific patterns of force can also be detected, such as when a user removes one earphone unit 104 from one ear while the other earphone unit 104 remains on the other ear, or when a user moves both earphone units 104 behind their ears, in front of them, or onto their neck. In some embodiments, the force sensor 320 may detect these changes individually. In other embodiments, a combination of sensors 220 may be used to determine the use (or other) state of the headset 100 at 410, 420.

[0052] For example, wearing detection can be easier or more accurate when each of the force sensor 320, IMU sensor 344, and proximity sensor 346 is used in a sensor fusion manner. The force sensor 320 provides sensing data related to the state of the headband 102 or the padding portion 106 (or both). The IMU sensor 344 provides sensing data related to the orientation of the headphones 100, such as whether the orientation is consistent with the headphones 100 being worn on the user's head and above the user's ears, or whether the headphones 100 are tilted, as if the headphones might be removed from the user's head and worn around the neck. The proximity sensor 346 provides sensing data about the relative position of each earpiece 104 (or one or more portions thereof) to the user's head, ears, or other body parts.

[0053] It can be seen how combining sensing data from these different sensor modalities can provide more accurate or additional information. For example, IMU sensor 344 can provide data indicating that the headset 100 is being worn around a user's neck, and if the data indicates that any force acting on the headband 102 is more consistent with neck wearing than with over-ear wearing, then data from force sensor 320 can confirm this. This can be further confirmed by considering data from proximity sensor 346 or by using proximity sensor 346 and IMU sensor 344. Therefore, these and other examples may include:

[0054]

[0055] Table 2: Example Features and Corresponding Sensors

[0056] Additional techniques can be used in conjunction with or in addition to these and other features. In one example of the “tilt to check” feature, two IMUs (one in each headphone unit 104) are used with machine learning (also discussed below) to help determine whether the headset 100 is on the user’s head, hanging around the user’s neck, on a table or other surface, etc. This information can also be used to check or determine whether other sensors 220 may be malfunctioning or inoperable.

[0057] As previously mentioned, these and other features may also be combined based on sensed wearing status or other activity. In one implementation, these features are pre-programmed and user-selectable. In other implementations, the user can customize which features(s) are implemented in different wearing statuses or sensed use.

[0058] These selections and customizations can be made on an application (“app”), a web-based application, or any other executable application framework running on a user device 330 communicatively coupled to the headset 100, such as a smartphone, smartwatch, or other wearable device (such as a fitness watch or tracker), tablet, e-reader, laptop, computer, or other computing device capable of interacting with the headset 100 (e.g., via Bluetooth) and hosting or presenting the app to the user. The term “user device” is used throughout this document for convenience but does not limit the actual characteristics, features, or composition of the device or any device that could embody the user device 330. The headset 100 is generally configured to provide bidirectional data communication with the user device 330 via a wired or wireless connection. In alternative embodiments, an electronic device having earphone units (e.g., earbuds) is configured to provide bidirectional data communication with the user device 330 via a wired or wireless connection.

[0059] The user interface can be configured to receive user input and provide output regarding the configuration and status of the headset 100, or alternatively, any electronic device characterized as a headphone unit. The user interface can allow for personalized system control and calibration, such as enabling the user to calibrate one or more active usage states by placing the headset in the desired position and recording sample force data. In an embodiment, the user equipment 330 may be associated with one or more user profiles, each of which may represent different feature processing requirements based on the detected usage state of the headset 100 or any electronic device including the headphone unit.

[0060] Processor 340 can automatically configure one or more features of the headset 100 based at least in part on the received sensor data. By processing sensing data related to the force applied from the headband 102 to the earcups 108 (from force sensor 320) and the orientation of the force (from IMU sensor 344 in one example), as well as detected capacitance or proximity (from proximity sensor 346), processor 340 can determine whether the headset 100 is on the user's head and thus can represent an active usage instance. In an alternative embodiment, processor 340 can process the detected sensor data from force sensor 320, IMU sensor 344, and proximity sensor 346 to determine whether the in-ear earbuds are in the user's ears and thus determine whether it is in an active usage state.

[0061] Specifically, the relative increase in force when the headband 102 extends (e.g., worn around or above the user's head) can be implemented within the processor 340 to identify active or inactive usage states and control the characteristics of the headset 100 accordingly. Active usage states may include any situation where the user might expect the headset to produce sound, provide noise cancellation, or generate other outputs. Inactive usage states may include one or more of the following: placing the headset 100 on a flat surface, folding it, storing it in a case, wearing it around the neck, placing it on a headset stand, or holding it by the user.

[0062] In an alternative implementation, processor 340 may identify differences in the forces applied to the outer surface of the electronic device, including the headphone unit (e.g., in-ear earbuds rather than over-ear headphones), to determine whether the usage state is active or inactive. As indicated above, an active usage state may include any situation in which the user might expect the electronic device to produce sound or provide noise cancellation functionality. An inactive usage state may include one or more of the following: placing the electronic device on a flat surface, folding it, storing it in a box or container, placing it on a stand, or holding it by the user.

[0063] The embodiments of this disclosure may optionally implement artificial intelligence (AI) or machine learning (ML) to better identify patterns or changes in sensor data associated with the wearing, use, and removal of the headset 100, headphones, or other electronic devices. Patterns or changes in forces, etc., sensed on the headset 100 during changes in use or wear status can be extracted manually or automatically using machine learning methods (such as, for example, convolutional neural networks) to generate training data that can be compared with data received during use. Thus, the ML model can be applied to labeled (supervised) sensor data (data representing known transitions in use status) by the processor 340. In the embodiments, unlabeled (unsupervised) sensor data may be used, although the accuracy and precision of the ML model will be relatively poor without additional training.

[0064] Through training, the ML model can better identify when differences in sensor data may indicate a change in usage state. In an implementation, the comparison process can be performed by calculating a similarity metric using a correlation or machine learning regression algorithm. For example, if the similarity of the "remove headphones" gesture is higher than a certain threshold (e.g., 75%, 90%, 95%, or 99% similarity), the matching process can determine that the received sensor data indicates a change in usage state, and the characteristics of the headphones 100 can be controlled accordingly, such as powering off the headphones 100.

[0065] This analysis can be improved by incorporating a feedback loop for classifying sensor data patterns for a specific user. Alternative implementations may include analyzing gestures related to headphones 100, earphones, or other electronic devices. For example, a "remove earbuds" gesture could exceed a certain threshold (as discussed in the previous example regarding headphones 100). As more comparisons of the received data and training data are completed, feedback on the accuracy of previous comparisons can be tracked to better identify future sensor data patterns.

[0066] In the implementation, the processor 340 may also implement one or more classifiers to consider parameters such as the type of electronic device (e.g., headset 100 versus headphones versus VR goggles) and the types of force sensor 320, IMU sensor 344, and proximity sensor 346.

[0067] During operation, a force applied to the outside of one or both earcups 108 by the headband 102 (or the user's head or ears) can automatically trigger a signal to power on the IMU sensor 344 and the proximity sensor 346. A force sensor 320, which may be located within one or both earcups 108 of the headset 100 or on or inside the earbuds, can work in conjunction with the IMU sensor 344 and the proximity sensor 346 to detect changes in force from the headband 102 to the earcups 108, wherein the detected force can be positive (e.g., expanding or extending the headband 102 / moving the earcups 108 away from each other) and negative (relaxing or retracting the headband 102 / moving the earcups 108 closer together). The detected change in force can automatically trigger the automatic power-on of electronic devices such as headphones 100, and can also automatically trigger control of features based on active usage status, which may include (in addition to those mentioned herein) establishing or disconnecting a Bluetooth connection between electronic devices such as headphones 100 and user equipment 330, enabling or disabling user input mechanisms (such as touch-based user interfaces) of electronic devices such as headphones 100, resuming or pausing media playback, changing playback volume, and switching between active noise cancellation level mode and transparency mode.

[0068] In implementation schemes, the AI ​​or ML system may be included in or communicatively coupled to processor 340 to further enable the headset 100 or user device 330 to respond to one or more settings or features based on a combination of data received from sensors (including sensor 220) and data received from one or more other sources (including user device 330 or other sources). This data may include location data, calendar data, proprioceptive data, health and wellness data, or other relevant user data. For example, one or more profiles may be created in response to data processing by the AI ​​system to reflect typical user behavior, activities, or preferred settings.

[0069] Therefore, embodiments of this disclosure provide usage status detection for electronic devices such as headphones or in-ear earbuds, and improve the robustness of usage status detection for these devices. By combining sensor modalities and data (e.g., wear detection and force) and efficiently processing this data, the use of these devices can be made more convenient, including by intelligently activating or deactivating various features. This may include deactivating one or more of the following: a touch UI, a mute microphone, a muted speaker, etc., when the device is handled but not actively worn. This eliminates the chance of unintentional triggering and the playback of music or other sounds unintentionally.

[0070] Other aspects of this disclosure enable devices such as headphones, in-ear earbuds, and any similar electronic devices to automatically power on or off, the electronic devices being reliable enough to eliminate the need for conventional power actuation methods (such as power buttons) on the electronic devices. Therefore, these devices can automatically turn on when a user puts them on and automatically turn off or become inactive in one or more ways when removed, thereby improving battery efficiency while simplifying the user experience. Additionally, the usage status detection described herein supports wireless integration and does not require any additional electronics or complexity within the electronic device.

[0071] refer to Figure 5 A flowchart of a method for automatically controlling one or more features of an electronic device (such as over-ear headphones) is depicted according to an embodiment. In the embodiment, this method can be implemented using headphones 100 or any electronic device (e.g., earbuds or in-ear headphones) having an earphone unit 104, as described below. Figure 5 The discussion uses headphones 100 as an example.

[0072] At 510, the wearing status of the headset 100 is detected, and at 520, one or more features of the headset 100 can be set based on the detected status. If the active usage status during wearing is determined, the headset 100 can be powered on and the selected features can be enabled. For example, a Bluetooth connection with a remote user device (330) can be established.

[0073] In a further iteration of the method, changes in the wearing status of the headphones 100 can be detected (return to 510). This may include the user pulling the earcups 108 apart so that they are pulled away from the ears and head when the user completely removes the headphones 100. Simultaneously, the IMU sensor can detect the motion orientation of the force (i.e., detect the “lowering” of the electronics such as the headphones 100) or the orientation of the headphones 100 in space. Furthermore, a proximity sensor can be configured to detect that the electronics such as the headphones 100 are not close to human skin for at least a preset threshold time period (e.g., 5 seconds). The sensors can send their corresponding sensed data to the processor 340 for processing, enabling sensor fusion in the processor 340 to more accurately determine whether the headphones 100 are being removed or no longer worn above the user's ears.

[0074] Based on this, one or more features of the headphones 100 can be set to an inactive state (at 520, the method iterates). This may include automatically disabling selected features based on the inactive usage state, including disabling certain features to save battery power and provide a better user experience, such as Bluetooth audio, touch UI, and pause playback. This may also (i.e., after a period of non-wearing) or alternatively include powering off the device.

[0075] Both active and inactive usage states can include multiple sub-states, each representing a specific type of use or inactivity. In one example, different sensor modalities may send different sensor data to processor 340 for processing, and the result may be an inactive sub-state where the headset 100 is determined to be "placed on a surface." This causes certain features of the headset 100 to be deactivated or paused, but the headset 100 is not completely powered off. In another example, processor 340 may determine, based on received sensor data, that the headset 100 is being held by a user. In this sub-state that indicates the user is about to use the headset 100, certain features may be selectively activated or prepared, allowing for a faster transition to an active usage state. For example, a Bluetooth connection to the user device 330 may be maintained, but playback may be paused. Other inactive or intermediate usage sub-states may include one or more of "folded," "stored," "hung around the neck," "on a stand," and "on a headset stand."

[0076] Figure 6A and 6B This is a flowchart of a method 600 for detecting usage status according to an implementation scheme. Storing the headphones 100 in a storage case triggers a force sensor 320 to detect slight forces applied to it. A contractile movement of the headphones 100 in a predetermined configuration triggers an IMU sensor 344, causing a processor 340 to power off the headphones 100. Similarly, if data from the IMU sensor 344 detects movement consistent with that of the headphones 100 in a handbag or backpack, the processor 340 can power off the headphones 100. Many other implementations are possible.

[0077] Various embodiments of this disclosure offer particular advantages for use with devices having “active” surfaces (e.g., external, touch-sensitive user interface features). Generally, the term “active” is defined as a surface configured to detect touch or contact by a user or defined input feature (e.g., a stylus), while the term “inactive” is defined as a surface that is non-capacitive, pressure-sensitive, or otherwise arranged to detect touch by transmitting signals.

[0078] The active surface allows the force sensor 320 to attempt to detect even seemingly small forces. The force sensor 320 can also be configured to detect forces applied to inactive surfaces, such as forces applied outward to the headband 102 when it extends or retracts (the forces may originate from the force generated when a user places the headphones 100 on their head) or forces applied outward to the ear tips of the earbuds when a user places the earbuds in their ears.

[0079] The active surface can be configured to be particularly sensitive to certain types of touch or materials. In one embodiment, the headset 100 may have at least one active surface arranged such that when a user places the headset 100 on their head, the active surface directly contacts the user's ears. This configuration enhances the ability of the processor 340 to acquire sensor data regarding the usage status of the headset 100. Here, the capacitive touch sensor of the active surface may be configured to detect human skin, such as the user's ears. Thus, the active surface may be located on one or both ear pads 110, elsewhere on one or both earcups 108, on the headband 102, etc. The processor 340 may combine the data from these sensors with data from the force sensor 320 regarding whether the headband 102 is extended or not to determine the usage status of the headset 100. If human skin is detected through the active surface, and if the force sensor 320 detects a force generated by the extension of the headband 102, the processor 340 may determine the active usage status and may automatically power on the headset 100 and enable selected features. If human skin is not detected on the active surface, or if the force sensor 320 does not detect the relevant force, the processor 340 can automatically determine the inactive usage state.

[0080] The following terms are part of this disclosure.

[0081] Clause 1: A system for improving the robustness of detecting the usage status of an electronic device, the system comprising: a plurality of sensors, the plurality of sensors including: at least one first sensor arranged to acquire data of a first type regarding the electronic device; and at least one second sensor arranged to acquire data of a second type regarding the electronic device, the second type of data being different from the first type of data; and at least one processor coupled to receive the first type of data from the at least one first sensor and the second type of data from the at least one second sensor, and from the first type of data and the second type of data to: determine a current usage status of the electronic device; compare the current usage status of the electronic device with a previous usage status; if the current usage status is different from the previous usage status, change the status of at least one feature of the electronic device based on the current usage status; and if the current usage status is the same as the previous usage status, maintain the status of at least one feature of the electronic device.

[0082] Clause 2: The system as described in Clause 1, wherein the plurality of sensors further includes at least one third sensor, the at least one third sensor being arranged to acquire a third type of data about the electronic device, the third type of data being different from the first type of data and the second type of data; and the at least one processor being coupled to receive the third type of data from the at least one third sensor and to determine the current usage state of the electronic device from the first type of data, the second type of data and the third type of data.

[0083] Clause 3: A system as described in Clause 2, wherein the first sensor is a force sensor, the second sensor is an inertial measurement unit (IMU) sensor, and the third sensor is a proximity sensor.

[0084] Clause 4: A system as described in any one of Clauses 1-3, wherein the electronic device comprises a headset having a first earphone unit and a second earphone unit, and the at least one first sensor comprises two inertial measurement units (IMUs), the first of the two IMUs being disposed within or on the first earphone unit, and the second of the two IMUs being disposed within or on the second earphone unit.

[0085] Clause 5: A system as described in any one of Clauses 1-4, wherein the at least one processor is configured to use machine learning to determine the current usage state of the electronic device from at least the first type of data.

[0086] Clause 6: A system as described in any one of Clauses 1-5, wherein: the current usage state of the electronic device is an active usage state, the previous usage state is an inactive usage state, and the state of changing at least one feature of the electronic device based on the current usage state includes activating at least one feature of the electronic device; or the current usage state of the electronic device is an inactive usage state, the previous usage state is an active usage state, and the state of changing at least one feature of the electronic device based on the current usage state includes deactivating at least one feature of the electronic device.

[0087] Clause 7: A system as described in any one of Clauses 1-6, wherein the electronic device includes the system such that the at least one processor and the plurality of sensors are arranged on or in the electronic device.

[0088] Clause 8: A system as described in Clause 6 or Clause 7, wherein: the electronic device includes a headset, the active use state being that the headset is worn on the user's head and above both ears, and the inactive state being that the headset is not worn above the user's ears; or the electronic device includes at least one earbud, the active use state being that the earbud is worn in the user's ear, and the inactive state being that the earbud is not worn in the user's ear.

[0089] Clause 9: A method for improving the robustness of detecting the usage status of an electronic device, the method comprising: acquiring first type of data about the electronic device; acquiring second type of data about the electronic device, the second type of data being different from the first type of data; determining a current usage status of the electronic device from the first type of data and the second type of data; comparing the current usage status of the electronic device with a previous usage status; if the current usage status is different from the previous usage status, changing the status of at least one feature of the electronic device based on the current usage status; and if the current usage status is the same as the previous usage status, maintaining the status of at least one feature of the electronic device.

[0090] Clause 10: The method of Clause 9, the method further comprising: acquiring third type of data about the electronic device, the third type of data being different from the first type of data and the second type of data; wherein determining the current usage state of the electronic device includes using the first type of data, the second type of data and the third type of data.

[0091] Clause 11: The method as described in Clause 10, wherein: acquiring a first type of data about the electronic device includes acquiring data on the forces acting on the electronic device; acquiring a second type of data about the electronic device includes acquiring data about the electronic device from at least one inertial measurement unit (IMU) sensor; and acquiring a third type of data about the electronic device includes acquiring data about the electronic device from at least one proximity sensor.

[0092] Clause 12: The method of any one of Clauses 9-11, wherein: determining the current usage state of the electronic device includes determining that the electronic device is in an active state; and determining that the current usage state is different from the previous usage state, such that changing at least one feature of the electronic device based on the current usage state includes activating at least one feature of the electronic device.

[0093] Clause 13: The method of any one of Clauses 9-12, wherein: determining the current usage state of the electronic device includes determining that the electronic device is inactive; and determining that the current usage state is different from the previous usage state, such that changing at least one feature of the electronic device based on the current usage state includes deactivating at least one feature of the electronic device.

[0094] Clause 14: The method as described in Clause 12 or Clause 13, wherein the electronic device includes a headset, the active state being that the headset is worn on the user's head and above both ears, and the inactive state being that the headset is not worn above the user's ears.

[0095] Clause 15: The method as described in Clause 12 or 13, wherein the electronic device includes at least one earbud, the active state being that the earbud is worn in the user's ear, and the inactive state being that the earphone is not worn in the user's ear.

[0096] Various embodiments of the systems, apparatus, and methods have been described herein. These embodiments are given by way of example only and are not intended to limit the scope of the claimed invention. Furthermore, it should be understood that the various features of the described embodiments can be combined in various ways to produce many additional embodiments. In addition, although various materials, sizes, shapes, configurations, and locations, etc., for the disclosed embodiments have been described, others besides those disclosed may also be used without exceeding the scope of the claimed invention.

[0097] Those skilled in the art will recognize that the subject matter of this disclosure may include fewer features than those shown in any single embodiment described above. The embodiments described herein are not intended to exhaustively represent all possible combinations of features of the subject matter of this disclosure. Therefore, the embodiments are not mutually exclusive combinations of features; rather, as will be understood by those skilled in the art, various embodiments may include combinations of different individual features selected from different individual embodiments. Furthermore, unless otherwise stated, elements described with respect to one embodiment may be implemented in other embodiments, even if not described in such embodiments.

[0098] While a dependent claim may refer in the claim to a specific combination with one or more other claims, other embodiments may also include a combination of the subject matter of that dependent claim with each of the other dependent claims, or a combination of one or more features with other dependent or independent claims. Such combinations are presented herein unless stated otherwise.

[0099] Any inclusion of the foregoing documents by reference is limited to ensuring that no subject matter contradicting the express disclosure herein is incorporated. Any inclusion of the foregoing documents by reference is further limited to ensuring that no claims included in the documents are incorporated herein by reference. Any inclusion of the foregoing documents by reference is also further limited to ensuring that any definitions provided in the documents are not incorporated herein by reference unless expressly included herein.

[0100] To interpret the claims, the explicit intent is to apply a means-plus-function interpretation, unless the specific terms "means for..." or "steps for..." are recorded in the claims.

Claims

1. A system for improving the robustness of electronic device usage status detection, the system comprising: Multiple sensors, the multiple sensors including: At least one first sensor, the at least one first sensor being arranged to acquire a first type of data about the electronic device, and At least one second sensor, the at least one second sensor being arranged to acquire a second type of data about the electronic device, the second type of data being different from the first type of data; and At least one processor, coupled to receive first-type data from at least one first sensor and second-type data from at least one second sensor, and from the first-type data and the second-type data: Determine the current usage status of the electronic device, and The current usage state of the electronic device is compared with its previous usage state. If the current usage state differs from the previous usage state, then the state of at least one feature of the electronic device is changed based on the current usage state. If the current usage state is the same as the previous usage state, then the state of at least one feature of the electronic device is maintained.

2. The system as claimed in claim 1, wherein: The plurality of sensors further includes at least one third sensor, the at least one third sensor being arranged to acquire a third type of data about the electronic device, the third type of data being different from the first type of data and the second type of data; and The at least one processor is coupled to receive the third type of data from the at least one third sensor, and to determine the current usage state of the electronic device from the first type of data, the second type of data, and the third type of data.

3. The system of claim 2, wherein the first sensor is a force sensor, the second sensor is an inertial measurement unit (IMU) sensor, and the third sensor is a proximity sensor.

4. The system of claim 1, wherein the electronic device comprises a headset having a first earphone unit and a second earphone unit, the at least one first sensor comprising two inertial measurement units (IMUs), the first of the two IMUs being disposed within or on the first earphone unit, and the second of the two IMUs being disposed within or on the second earphone unit.

5. The system of claim 1, wherein the at least one processor is configured to use machine learning to determine the current usage state of the electronic device from at least the first type of data.

6. The system of claim 1, wherein: The current usage state of the electronic device is an active usage state, the previous usage state is an inactive usage state, and the state of changing at least one feature of the electronic device based on the current usage state includes activating at least one feature of the electronic device; or The current usage state of the electronic device is an inactive usage state, the previous usage state is an active usage state, and the state of changing at least one feature of the electronic device based on the current usage state includes deactivating at least one feature of the electronic device.

7. The system of claim 1, wherein the electronic device includes the system, such that the at least one processor and the plurality of sensors are arranged on or in the electronic device.

8. The system of claim 6, wherein: The electronic device includes a headset, wherein the active use state is when the headset is worn on the user's head and above both ears, and the inactive state is when the headset is not worn above the user's ears; or The electronic device includes at least one earplug, the active use state being when the earplug is worn in the user's ear, and the inactive state being when the earplug is not worn in the user's ear.

9. A method for improving the robustness of electronic device usage status detection, the method comprising: Obtain data of the first type regarding the electronic device; Acquire a second type of data about the electronic device, the second type of data being different from the first type of data; The current usage status of the electronic device is determined from the first type of data and the second type of data; The current usage state of the electronic device is compared with its previous usage state; If the current usage state is different from the previous usage state, then the state of at least one feature of the electronic device is changed based on the current usage state; as well as If the current usage state is the same as the previous usage state, then the state of at least one feature of the electronic device is maintained.

10. The method of claim 9, further comprising: Acquire a third type of data about the electronic device, which is different from the first type of data and the second type of data; Determining the current usage state of the electronic device includes using the first type of data, the second type of data, and the third type of data.

11. The method of claim 10, wherein: Acquiring a first type of data about the electronic device includes acquiring data on the forces acting on the electronic device; Acquiring a second type of data about the electronic device includes acquiring data about the electronic device from at least one inertial measurement unit (IMU) sensor; and Acquiring a third type of data about the electronic device includes acquiring data about the electronic device from at least one proximity sensor.

12. The method of claim 9, wherein: Determining the current usage state of the electronic device includes determining that the electronic device is in an active state; and Determining that the current usage state is different from the previous usage state, such that changing at least one feature of the electronic device based on the current usage state includes activating at least one feature of the electronic device.

13. The method of claim 9, wherein: Determining the current usage status of the electronic device includes determining that the electronic device is inactive; and Determining that the current usage state is different from the previous usage state, such that a state in which at least one feature of the electronic device is changed based on the current usage state includes disabling at least one feature of the electronic device.

14. The method of claim 12, wherein the electronic device includes headphones, the active state being that the headphones are worn on the user's head and above the ears, and the inactive state being that the headphones are not worn above the user's ears.

15. The method of claim 12, wherein the electronic device includes at least one earplug, the active state being that the earplug is worn in the user's ear, and the inactive state being that the earplug is not worn in the user's ear.

Images