A method for detecting wear and wireless earphones

By combining sensor data and audio data for feature extraction, the problem of misjudgment in wireless headphone wear detection has been solved, achieving higher accuracy and reliability, and improving user experience and power utilization efficiency.

CN117793592BActive Publication Date: 2026-11-17HUAWEI DEVICE CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202211157191.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-22
Publication Date
2026-11-17
Estimated Expiration
2042-09-22

AI Technical Summary

Technical Problem

Existing methods for detecting wear in wireless earphones are prone to misjudgment, resulting in low accuracy, which affects user experience and increases power consumption.

Method used

By combining sensor data and audio data for feature extraction, and acquiring sensor and audio features through synchronous or asynchronous acquisition, the wearing status is determined by the difference in signal energy under different states.

Benefits of technology

It improves the accuracy and reliability of wear detection, reduces the probability of false triggers, and ensures a smooth user experience and power efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117793592B_ABST
    Figure CN117793592B_ABST
Patent Text Reader

Abstract

This application provides a wearing detection method and a wireless earphone, the wireless earphone including an earphone body and an earphone case, relating to the field of electronic device technology. The wireless earphone can, in response to a trigger operation, acquire sensor features corresponding to sensor data and audio features corresponding to audio data. The trigger operation includes a user touch operation on the earphone body, a user separation operation between the earphone body and the earphone case, or a user touch and move operation on the earphone body. The wireless earphone can then fuse the sensor features and audio features. Based on the fused sensor features and audio features, it determines the current wearing status of the wireless earphone. This provides the user with more accurate wearing detection results, meeting the user's operational needs in multiple scenarios.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of electronic device technology, and in particular to a method for detecting wear and a wireless headset. Background Technology

[0002] With the rapid development of artificial intelligence technology, people are increasingly pursuing smarter, more functional, and more human-centered lifestyles. Wireless headphones, as the primary means of transmitting audio signals in wearable devices, are widely used. Typically, when a user wears wireless headphones, the headphones can communicate with the terminal device using wireless communication technology, allowing the user to hear the corresponding audio information during the communication.

[0003] Typically, wireless earbuds need to incorporate a wear detection device to determine their current status and initiate or deactivate the earbuds accordingly. For example, multiple sensors can be placed within the earbuds to determine whether they are being worn based on the signals collected by these sensors. Only when the earbuds are being worn can they perform their corresponding tasks.

[0004] However, due to the aforementioned method of deploying multiple sensors, wireless earbuds can be misidentified as being worn in many scenarios. For example, when a user handles the earbuds, pinches them to remove them from their case, or picks them up from a table, accidental touches on the sensors can lead to misidentification. This results in low accuracy in wear detection, impacting user experience and increasing the power consumption of the wireless earbuds. Summary of the Invention

[0005] This application provides a wearing detection method and a wireless earphone. By combining collected sensor data and audio data, the current wearing status of the wireless earphone can be determined more accurately. This ensures the timeliness and effectiveness of wearing detection, providing users with a smooth usage process and improving the user experience.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] Firstly, a wearing detection method is provided, applied to wireless earphones. The wireless earphones include an earphone body and an earphone case. The method includes: the wireless earphones, in response to a trigger operation, acquiring sensor features corresponding to sensor data and audio features corresponding to audio data. The trigger operation includes a user touch operation on the earphone body, a user separation operation between the earphone body and the earphone case, or a user touch and move operation on the earphone body. Then, the wireless earphones fuse the acquired sensor features and audio features to obtain fused features. Finally, the method determines whether the wireless earphones are being worn based on the fused features.

[0008] The aforementioned wearing detection method allows for a combined determination of the current wearing status of wireless headphones based on fused features, integrating both sensor and audio characteristics. This avoids misjudgments that can occur when relying solely on sensor features, providing users with more accurate wearing detection results. It reduces the probability of accidental triggering by users and improves the accuracy and reliability of headphone wearing detection.

[0009] In one possible implementation of the first aspect, the process of the wireless earphone acquiring the sensing features corresponding to the sensor data and the audio features corresponding to the audio data includes: the wireless earphone simultaneously acquiring sensor data and audio data. Then, feature extraction is performed on the sensor data and audio data to obtain the sensing features and audio features. In this implementation, the wireless earphone simplifies and saves data processing time by simultaneously acquiring sensor data and audio data. This avoids long waiting times for users and improves the user experience.

[0010] In one possible implementation of the first aspect, during the process of acquiring sensor features corresponding to sensor data and audio features corresponding to audio data, the wireless earphone can first collect sensor data and extract features from the sensor data to obtain sensor features. Then, based on the sensor features, when a preset condition is met, audio data is collected. Finally, features are extracted from the audio data to obtain audio features. The preset condition includes that the feature value corresponding to the sensor feature is within a first preset range. Thus, when the feature value corresponding to the sensor feature is within the first preset range, the wireless earphone is in motion. In this implementation, when the feature value corresponding to the sensor feature is within the first preset range, it can be determined that the wireless earphone is moving towards the human body. The wireless earphone then controls the audio module to collect audio data and extract audio features from the audio data. This allows for a more accurate determination of whether the wireless earphone is being worn based on the fused audio features and sensor features. Furthermore, pre-determining the feature value based on the sensor feature eliminates the need for simultaneous collection of sensor data and audio data, reducing the power consumption of the wireless earphone.

[0011] In one possible implementation of the first aspect, during the process of acquiring sensor data, the wireless earphone can pre-power on the microphone after the data acquisition duration is greater than or equal to a preset duration. Subsequently, the wireless earphone acquires audio data based on the already powered-on microphone. In this implementation, by pre-powering on the microphone, the wireless earphone ensures that the microphone is already operating stably when audio acquisition is needed, thus enabling direct audio data acquisition. This reduces the latency of the wear detection process.

[0012] In one possible implementation of the first aspect, the sensor data and audio data include signals during the movement of the wireless earphone and its contact with a human body; the audio data includes a preset audio signal played by a speaker, and / or an audio signal generated during the contact of the earphone body with a human body. The audio signal generated during the contact of the earphone body with a human body includes a first audio signal corresponding to a change in air pressure caused by the wireless earphone contacting the ear, and a second audio signal corresponding to friction generated by the wireless earphone contacting the ear.

[0013] In one possible implementation of the first aspect, the sensor data includes posture data, motion data, distance data, and signal data throughout the entire process of the earphone being put into the ear; the sensor data is used to determine the proximity of the human body to the wireless earphone and / or the touch action of the human body to the wireless earphone.

[0014] In one possible implementation of the first aspect, the signal energy of the preset audio signal corresponding to the wireless earphone in the wearing state is greater than the signal energy of the preset audio signal corresponding to the wireless earphone in the non-wearing state, and the audio characteristics corresponding to the different signal energies are different.

[0015] In this feasible approach, the audio data differs depending on the state. Audio data when the headphones move towards the body includes audio data from the open field when the headphones are not in the ear. Audio data when the headphones touch the ear includes audio data from the closed field when the headphones are in the ear. Both open and closed field audio data include the preset audio signal played by the speaker. In an open field, most of the audio signal dissipates into the air, resulting in lower energy of the audio signal picked up by the microphone. In a closed field, the dissipation of audio signal energy is less than in an open field, and the signal energy of the preset audio signal played by the speaker picked up by the microphone is greater than that picked up by the microphone in an open field. Therefore, the signal energy of the audio signal in a closed field is greater than that in an open field. Thus, the current wearing status of the wireless headphones can be determined based on the different signal energies corresponding to different audio features in the fusion characteristics.

[0016] In one possible implementation of the first aspect, the wireless earphone determines whether it is being worn based on sensing features and audio features, including: determining whether the wireless earphone is being worn based on sensing features and target audio features, wherein the time-domain clipping feature of a first audio signal represented by the target audio features is greater than a first preset threshold and the Mel-frequency cepstral coefficient of a second audio signal in the mid-to-low frequency band is greater than a second preset threshold. Wherein, the signal energy of the preset audio signal represented by the target audio features is greater than a third preset threshold. Therefore, in this possible implementation, the wireless earphone can determine whether it is being worn based on sensing features and target audio features. That is, after the target audio features meet the constraints, the subsequent determination of whether the earphone is being worn can be made in conjunction with the sensing features.

[0017] In one possible implementation of the first aspect, the wireless earphone can assign weights to the sensor features and audio features during the fusion process. Then, the weighted sensor features and audio features are concatenated to complete the feature fusion. In this implementation, by concatenating the sensor features and audio features based on weights, redundant information in the sensor features and audio features can be eliminated, making the resulting fused features more explicit and thus improving the accuracy of the judgment on the fused features.

[0018] In one possible implementation of the first aspect, during the process of determining whether the wireless earphone is being worn based on fusion features, the fusion feature value corresponding to the fusion feature can be compared with a second preset range. Then, if the fusion feature value is within the second preset range, it is determined that the wireless earphone is being worn. If the fusion feature value is outside the second preset range, it is determined that the wireless earphone is not being worn. In this implementation, when the fusion feature value is within the second preset range, the sensing feature in the fusion feature indicates that the contact area between the earphone cap and the human body is greater than a preset value, and the audio feature in the fusion feature indicates that the earphone is in a closed field and is inserted into the ear. Therefore, in scenarios such as when the user plays with the earphone or covers the earphone, even if the sensor data changes significantly due to the user touching the sensor with their finger, the fusion feature value will not be within the second preset range, thus failing to determine that the earphone is currently being worn. This application embodiment can more accurately determine whether the wireless earphone is being worn by combining audio data, avoiding the situation where the earphone is mistakenly turned on based on a single sensor data.

[0019] In one possible implementation of the first aspect, after determining whether the wireless earphone is in a wearing state based on the fusion features, the method further includes: when the wireless earphone is in a wearing state, controlling the wireless earphone to start a corresponding working task; and when the wireless earphone is not in a wearing state, controlling the wireless earphone not to start or terminate the corresponding working task.

[0020] In a second aspect, a wireless earphone is provided, comprising: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the wireless earphone to perform the wear detection method as described in any one of the first aspects above.

[0021] Thirdly, a wearable device is provided, comprising: one or more processors, a memory, and a communication interface; wherein the memory and the communication interface are coupled to the processor; the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, causes the wireless earphone to perform the wear detection method as described in any one of the first aspects above.

[0022] Fourthly, a computer-readable storage medium is provided that stores instructions which, when executed on a computer, enable the computer to perform the wear detection method described in any one of the first aspects.

[0023] The technical effects of any of the design methods in the second to fourth aspects can be found in the technical effects of different design methods in the first aspect, and will not be repeated here. Attached Figure Description

[0024] Figure 1 This is a schematic diagram illustrating a user's use of a wearable device, provided in an embodiment of this application.

[0025] Figure 2 A schematic diagram of the hardware structure of a wireless earphone provided in an embodiment of this application;

[0026] Figure 3 A schematic diagram of the software structure of a wireless earphone provided in an embodiment of this application;

[0027] Figure 4 This application provides a schematic diagram of the structure of a system library in a wireless headset.

[0028] Figure 5 A schematic flowchart of a wear detection method provided in an embodiment of this application;

[0029] Figure 6 A schematic flowchart illustrating another wear detection method provided in an embodiment of this application;

[0030] Figure 7 A schematic flowchart illustrating another wear detection method provided in an embodiment of this application;

[0031] Figure 8This application provides a schematic diagram of a user wearing headphones in an embodiment of the present application.

[0032] Figure 9 This is a schematic diagram of the structure of a wear detection device provided in an embodiment of this application;

[0033] Figure 10 This is a schematic diagram of the structure of a wearable device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the words "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0035] Furthermore, the network architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0036] Currently, with the rapid development of artificial intelligence technology, wearable devices are becoming increasingly common. Many users may wear wearable devices daily for extended periods. These wearable devices are small and convenient, allowing users to listen to audio, make voice calls, and even sing along with others.

[0037] In the embodiments of this application, wearable devices may include wireless headphones, smart glasses, augmented reality (AR) glasses, virtual reality (VR) glasses, VR headsets, and AR headsets, etc.

[0038] Figure 1 The illustration depicts a scenario where a user is using a wearable device. This application uses a wireless headset as an example of a wearable device; see [link / reference]. Figure 1 In the example (A), the wireless earphones eliminate the need for wired connections and typically consist of two earphone bodies 12 (e.g., a left earphone body 121 and a right earphone body 122). Both the left and right earphone bodies 121 and 122 include an earphone stem 123 and an earphone cap 124, with the cap 124 positioned at the top of the stem 123. The wireless earphones also come with a charging case 11, which serves both charging and storage functions. The case 11 provides power to both earphone bodies 12 and provides storage. When using the wireless earphones, users usually need to remove the left and right earphone bodies 121 and 122 from the case 11 and then place them on their ears.

[0039] The wireless earphones in this application embodiment can be Bluetooth earphones, such as over-ear Bluetooth earphones, neckband Bluetooth earphones, in-ear Bluetooth earphones, ear-hook Bluetooth earphones, and TWS earphones, etc.

[0040] See Figure 1 In section (B), to enable users to interact with wearable devices more conveniently and accurately, this application will use wireless earphones as an example for specific explanation. After the user has worn both earphone bodies 12, the user can use the terminal device 20 to control the two earphone bodies 12. This allows the user to interact indirectly with the wireless earphones. For example, the terminal device 20 connects the two earphone bodies 12 via Bluetooth, allowing the user to listen to notifications, reminders, etc., while wearing the earphone bodies 12. The user can also listen to music, audiobooks, and control intelligent voice assistants.

[0041] Meanwhile, when the earbuds 12 are low on power, the user simply places them into the charging case 11. The earbuds 12 automatically disconnect from the terminal device 20 via Bluetooth, and the charging case 11 then charges them. During the connection process between the terminal device 20 and the earbuds 12, the terminal device 20 can prioritize connecting to the left earbud 121, which then connects to the right earbud 122 via Bluetooth, enabling wireless separation of the left and right channels. Similarly, the terminal device 20 can prioritize connecting to the right earbud 122, which then connects to the left earbud 121 via Bluetooth. Furthermore, either earbud 12 can operate independently. For example, when the right earbud 122 is not connected to the left earbud 121, the left earbud 121 can revert to mono sound quality, allowing for independent use of a single earbud.

[0042] For example, the terminal device 20 in this application embodiment can be a portable computer (such as a mobile phone), tablet computer, laptop computer, personal computer (PC), augmented reality (AR) / virtual reality (VR) device, in-vehicle computer, etc. This application embodiment does not impose any special restrictions on the specific form of the terminal device 20.

[0043] Based on the above practical scenarios, the prerequisite for wearable devices such as wireless earphones to work in conjunction with the aforementioned terminal device 20 and provide various services to users is that the user wears the wearable device. Taking wireless earphones as an example, the earphone body 12 (hereinafter referred to as earphones) is taken out of the earphone case by the user, and the user wears the earphones in their ears. When the earphones are taken out of the earphone case by the user, it is necessary to detect the current wearing status of the earphones in a timely manner in order to activate the relevant functions of the earphones (such as Bluetooth function, call function, and active noise cancellation function). Similarly, when the earphones are taken out of the user's ears, it is also necessary to detect the current wearing status of the earphones in a timely manner in order to deactivate the above functions of the earphones in a timely manner to save the earphones' battery power. It can be seen that earphone wearing status detection is particularly important for earphones.

[0044] Typically, before performing a task, headphones use an internal wear detection device to determine their wearing status and then start or stop working based on the detection result. For example, the wear detection device may include multiple sensors, and the headphones can determine whether they are being worn based on the sensor signals collected by these sensors.

[0045] Understandably, if the earphones are currently being worn, they will begin performing the corresponding tasks. If the wireless earphones are not currently being worn, they will neither begin nor stop performing the corresponding tasks. Here, "worn" refers to the state when the wireless earphones are worn in the corresponding position on the ear. "Not worn" refers to the state when the wireless earphones are not worn in the corresponding position on the ear.

[0046] See also Figure 1 In step (B), it can be understood that the user has already placed the earphones in the appropriate position on their ear. Thus, once it is confirmed that the user has placed the earphones in their ear, the earphones can perform functions such as audio playback, message notifications, and voice calls.

[0047] However, the aforementioned method of using sensors to detect the current status of the earphones can be misinterpreted as being worn in many scenarios. For example, handling the earphones, pinching them to remove them from the case, or picking them up from a table can all cause misjudgments due to accidental touches on the sensor. This results in low accuracy in wear detection, affecting the user experience and increasing the power consumption of the wireless earphones.

[0048] Furthermore, in scenarios that affect sensor stability, such as when headphones are exposed to temperature drift or wet environments, the headphones may experience changes in the parameters of the internal semiconductor devices if subjected to external temperature, humidity, electromagnetic interference, and interference from the sensor conditioning circuit. Consequently, the sensor's final sensor signal may become abnormal, leading to a higher false detection rate in wear detection and reducing its reliability.

[0049] As can be seen from the above, current headphone wear detection methods suffer from inaccurate detection results and increased headphone power consumption in many scenarios, thus affecting the user experience.

[0050] Based on the above, this application provides a wear detection method that can be applied to wearable devices for wear detection. Taking headphones as an example, the headphones include sensors for collecting sensor data, speakers for transmitting audio data, and microphones for receiving audio data. After receiving an activation detection command, the headphones can acquire the sensor data collected by the sensors and combine it with the audio data collected by the microphone to more accurately determine the current wear detection result. Then, the headphones activate or deactivate the corresponding task based on the wear detection result. Specifically, the headphones can extract the sensing features corresponding to the sensor data and the audio features corresponding to the audio data based on the sensor data and audio data. The sensor data and audio data include signals during headphone movement and contact with the ear. In this way, the headphones can fuse the extracted sensing features and audio features to determine the current wear detection result based on the fused features. Finally, the headphones activate, deactivate, or deactivate the corresponding task based on the wear detection result.

[0051] The wearing detection method provided in this application can determine the current wearing state of the headphones based on the fused sensor features and audio features, thereby providing users with more accurate wearing detection results and meeting their operational needs in multiple scenarios. For example, when a user is playing with the headphones in their hand, pinching the headphones to remove them from the case, or picking them up from a table. The headphones can accurately determine whether they are currently not being worn by combining the characteristics of the current scene's sensor features and audio features, thus eliminating the need to activate corresponding tasks. After the user places the headphones in the appropriate position on their ear, the headphones can determine the current state of the headphones as being worn based on the sensor data and audio data collected during the process of the headphones moving to and touching the ear, and only then will the headphones activate corresponding tasks. This improves the accuracy of wearing detection, ensures its timeliness and effectiveness, provides users with a smooth usage process, and enhances the user experience.

[0052] Taking wireless headphones as an example, Figure 2 A schematic diagram of the hardware structure of the wireless earphone 100 is shown. See also... Figure 2The wireless earphone 100 provided in this application embodiment may include a processor 110, an internal memory 120, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a wireless communication module 150, an audio module 160, a speaker 160A, a microphone 160B, a sensor module 170, buttons 180, and an indicator 190, etc. The sensor module 170 may include a capacitive sensor 170A, a gyroscope sensor 170B, a barometric pressure sensor 170C, a proximity sensor 170D, a proximity light sensor 170E, a fingerprint sensor 170F, a temperature sensor 170G, a touch sensor 170H, an ambient light sensor 170I, and a bone conduction sensor 170J, etc. The processor 110, internal memory 120, antenna, wireless communication module 150, audio module 160, speaker 160A, microphone 160B, sensor module 170, and buttons 180 may be located within the earphone. The aforementioned Universal Serial Bus interface 130, charging management module 140, power management module 141, battery 142, and indicator 190 can be located in the headphone case.

[0053] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the wireless earphone 100. In other embodiments of this application, the wireless earphone 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0054] The processor 110 may include one or more processing units, such as a controller, memory, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0055] The controller can serve as the central nervous system and command center of the wireless headset 100. The controller can generate operation control signals based on instruction operation codes and timing signals to control the fetching and execution of instructions.

[0056] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0057] Simultaneously, the memory can also be used to store program code, such as program code for establishing virtual or physical connections between the wireless headset 100 and the terminal device, switching the physical connection with the terminal device, handling audio services of the terminal device (e.g., music playback, making / receiving calls), and program code for charging the wireless headset 100 and wirelessly pairing the wireless headset 100 with other terminal devices. The memory can also be used to store other information, such as the priority of the terminal device 20.

[0058] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0059] The I2C interface can be used to implement touch functionality. For example, the processor 110 can couple to the touch sensor 170H through the I2C interface, enabling the processor 110 and the touch sensor 170H to communicate via the I2C bus interface, thus realizing the touch functionality of the wireless headset 100.

[0060] The I2S interface can be used for audio communication. The audio module 160 can transmit audio signals to the wireless communication module 150 through the I2S interface, enabling the function of answering telephone calls through the wireless headset 100.

[0061] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. The audio module 160 can also transmit audio signals to the wireless communication module 150 via the PCM interface, enabling the function of answering phone calls through the wireless headset 100.

[0062] The UART interface is typically used to connect the processor 110 and the wireless communication module 150. The audio module 160 can transmit audio signals to the wireless communication module 150 via the UART interface, enabling music playback via the wireless headphones 100.

[0063] The GPIO interface can be configured via software. The GPIO interface can be used to connect the processor 110 to the wireless communication module 150, audio module 160, sensor module 170, etc.

[0064] USB port 130 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge the wireless headphones 100.

[0065] In some embodiments, the wireless earphone 100 may further include an input / output interface. The input / output interface can be used to provide any wired connection between the earphone and the earphone case. In some embodiments, the input / output interface may be an electrical connector. For example, when the earphone is placed in the cavity of the earphone case, the earphone can be electrically connected to the earphone case via the electrical connector. After the electrical connection is established, the earphone case can charge the earphone. After the electrical connection is established, the earphone can also communicate data with the earphone case. For example, the earbud processor 110 can receive a pairing command from the earphone case via the electrical connection. This pairing command instructs the processor 110 to activate the wireless communication module 150, thereby enabling the earphone to pair with the terminal device 20 using a corresponding wireless communication protocol (such as Bluetooth).

[0066] Of course, the aforementioned headphones may also not include input / output interfaces. In this case, the headphones can achieve charging or data communication functions based on the Bluetooth connection established between the headphones and the headphone case via the aforementioned wireless communication module 150.

[0067] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the wireless headset 100. In other embodiments of this application, the wireless headset 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0068] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the wireless headset 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0069] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, internal memory 120, external memory, and wireless communication module 150. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0070] The wireless communication function of the wireless earphone 100 can be achieved through an antenna and a wireless communication module 150. The antenna is used to transmit and receive electromagnetic wave signals.

[0071] The wireless communication module 150 can provide solutions for wireless communication applications on the wireless headset 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via an antenna, modulates and filters the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 150 can also receive signals to be transmitted from the processor 110, modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0072] The wireless communication module 150 can be a Bluetooth chip. The wireless headset 100 can pair with the Bluetooth chip of the terminal device 20 and establish a wireless connection through this chip, enabling wireless communication and service processing between the wireless headset 100 and other terminal devices 20. This wireless connection can be a physical connection or a virtual connection. Typically, the Bluetooth chip supports basic rate (BR) / enhanced data rate (EDR) Bluetooth and Bluetooth Low Energy (BLE). For example, it can send / receive page messages and BLE broadcast messages.

[0073] The internal memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc. The data storage area may store data (such as audio data) generated during the use of the wireless headset 100.

[0074] The wireless earphone 100 can achieve audio functions such as music playback and recording through the audio module 160, speaker 160A, microphone 160B, and application processor.

[0075] The audio module 160 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 160 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 160 may be located in the processor 110, or some functional modules of the audio module 160 may be located in the processor 110.

[0076] The speaker 160A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The wireless headset 100 can listen to music or make hands-free calls through the speaker 160A.

[0077] Microphone 160B, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 160B, inputting the sound signal into microphone 160B. Wireless headset 100 may have at least one microphone 160B. In some embodiments, wireless headset 100 may have two microphones 160B, such as a main microphone and a feedback microphone. The main microphone can be used to pick up the user's corresponding sound signal. The feedback microphone can be used to collect sound signals corresponding to the user and the surrounding environment, and to implement noise reduction. In the embodiments of this application, the wireless headset can extract audio features based on the audio data collected by the feedback microphone, and determine whether the wireless headset is being worn based on the audio features.

[0078] In other embodiments, the wireless earphone 100 may also be equipped with three, four or more microphones 160B to collect sound signals, reduce noise, identify the sound source, and achieve directional recording functions, etc.

[0079] A capacitive sensor 170A is used to sense the capacitance signal between the human body and the sensor. The wireless earphone 100 determines the proximity of the human body based on the change in capacitance. When the earphone is close to the user's ear, the wireless earphone 100 can detect the proximity action based on the capacitive sensor 170A.

[0080] The gyroscope sensor 170B can be used to determine the motion attitude of the wireless earphone 100. In some embodiments, the angular velocity of the wireless earphone 100 about three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 170B.

[0081] A barometric pressure sensor 170C is used to measure barometric pressure. In some embodiments, the wireless earphone 100 uses the barometric pressure value measured by the barometric pressure sensor 170C to detect whether there is an object near the wireless earphone 100, thereby determining whether the wireless earphone 100 is being worn by a user. When it is determined that the wireless earphone 100 is being worn, the wireless earphone 100 can turn on the speaker and perform corresponding tasks.

[0082] The distance sensor 170D can be used to measure distance. The wireless earphone 100 can measure distance using infrared or laser. In some embodiments, it can be used to determine whether the wireless earphone 100 is being worn by a user based on the measured distance.

[0083] The proximity sensor 170E may include, for example, a light-emitting diode (LED) and a photosensor. The wireless earphone 100 uses the photodiode to detect infrared reflected light from nearby objects. The proximity sensor 170E may be located at the bottom of the earphone stem 123. When sufficient reflected light is detected, it can determine that an object is near the wireless earphone 100. When insufficient reflected light is detected, it can determine that no object is near the wireless earphone 100. The wireless earphone 100 can use the proximity sensor 170E to detect whether the wireless earphone 100 is close to the user's ear for subsequent activation of corresponding tasks. The wireless earphone 100 can also use the proximity sensor 170E to detect whether the earphone is detached from the earphone case.

[0084] The ambient light sensor 170I is used to sense the ambient light intensity. The wireless earphone 100 can adaptively adjust some audio parameters (such as volume) based on the sensed ambient light intensity. The fingerprint sensor 170F is used to collect fingerprints. The wireless earphone 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, app access lock, etc.

[0085] Temperature sensor 170G is used to detect temperature. In some embodiments, wireless earphone 100 uses the temperature detected by temperature sensor 170G to execute a temperature processing strategy.

[0086] Touch sensor 170H can detect user touch operations such as single clicks, double clicks, multiple clicks, long presses, and heavy pressure. It can also perform fingerprint recognition for user authentication in payment transactions and other business scenarios. Touch sensor 170H is used to detect touch operations applied to or near it. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Touch sensor 170H can also be placed on the surface of wireless earphone 100.

[0087] The bone conduction sensor 170J can acquire vibration signals. In some embodiments, the bone conduction sensor 170J can also be integrated into a bone conduction headset. The audio module 160 can analyze the voice signal based on the vibration signal of the sound-vibrating bone block acquired by the bone conduction sensor 170J to realize the voice function and receive the user's voice commands. The wireless headset 100 can also perform voice authentication, etc., based on the user's voice signal acquired by the bone conduction sensor 170J.

[0088] Button 180 can be a mechanical button or a touch button. The wireless headset 100 can receive button input and generate key signal inputs related to user settings and function control of the wireless headset 100. Button 180 is used to trigger operations such as powering on, powering off, pausing, playing, recording, initiating pairing, and resetting the wireless headset 100.

[0089] Indicator 190 can be an indicator light, which can be used to indicate the status of wireless earphone 100, such as battery level, incoming / outgoing calls, and pairing mode.

[0090] It is understood that the structure illustrated in the embodiments of this application does not constitute a specific limitation on the wireless earphone 100. It may have more than Figure 2 The number of components shown may be more or less, two or more components may be combined, or different component configurations may be present. For example, the wireless headset 100 may also include a dust filter (which can be used with the earpiece) component. Figure 2 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing or application-specific integrated circuits.

[0091] The software system of the wireless earphone 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered Android system as an example to exemplify the software structure of the wireless earphone 100.

[0092] Figure 3This is a software structure block diagram of the wireless earphone 100 according to an embodiment of this application.

[0093] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0094] The application layer can include a series of application packages.

[0095] like Figure 3 As shown, the application package can include applications such as calling, location, Bluetooth, and music.

[0096] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0097] like Figure 3 As shown, the application framework layer can include content providers, phone managers, resource managers, etc.

[0098] The content provider stores and retrieves data, making it accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phonebook entries, etc. The phone manager provides communication functionality for the wireless headset 100, such as managing call status (including connection and disconnection). The resource manager provides various resources to the application, such as localized strings, layout files, audio files, etc.

[0099] The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

[0100] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0101] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0102] The system library can include multiple functional modules. For example: sensor module, sensor feature extraction module, audio module, audio feature extraction module, feature fusion module, and fused feature decision module.

[0103] The kernel layer is the layer between hardware and software. The kernel layer includes at least audio drivers and sensor drivers, among others.

[0104] See Figure 4 The following description, using a wireless headset as an example, illustrates the workflow of the sensor module 41, the sensor feature extraction module 42, the audio module 43, the audio feature extraction module 44, the feature fusion module 45, and the fusion feature decision module 46.

[0105] In some embodiments, the sensor feature extraction module 42, audio feature extraction module 44, feature fusion module 45 and fusion feature decision module 46 described above can also be integrated into a single functional module, which can be located in the terminal device 20 or in the server.

[0106] The sensor module 41 may include one or more of the following: an IMU, a gyroscope, a gravity sensor, an accelerometer, a magnetometer, and other sensors (such as temperature sensors, distance sensors, pressure sensors, and capacitance sensors). The sensor module 41 can collect posture data, motion data, distance data, and signal data throughout the entire process of the earphone being put into the ear. The sensor feature extraction module 42 can extract the sensor features corresponding to the earphone moving to the ear and / or the earphone touching the ear from the sensor data collected by the sensor module 41. Examples include capacitance signal values ​​and the contact area between the earphone and the ear.

[0107] Furthermore, the audio module 43 may include a speaker and a microphone, and can collect audio data during the entire process of a user putting the headphones into their ear, such as preset audio signals transmitted by the speaker, including sound waves and ultrasound. Other examples include the audio signal corresponding to the earphone rubbing against the ear canal and the audio signal corresponding to the air pressure change caused by the earphone contacting the ear canal. The audio feature extraction module 44 can extract the friction and impact sound features of the earphone insertion action and / or the audio features corresponding to the closed field when the headphones are in the ear from the audio data collected by the audio module 43. Examples include waveform clipping features, signal envelope features, frequency energy features of frequency domain signals, and Mel-frequency cepstral coefficients corresponding to low-frequency or mid-frequency bands.

[0108] Next, the feature fusion module 45 concatenates the sensor features and audio features extracted by the sensor feature extraction module 42 and the audio feature extraction module 44 to obtain the fused features. Finally, the fused feature decision module 46 determines the current state of the headphones based on the fused features concatenated by the feature fusion module 45, obtaining the wearing detection result. The wearing detection result includes both the wearing state and the not-wearing state of the headphones. Based on the wearing detection result, it is determined whether the headphones should start or stop their corresponding operation.

[0109] In the above Figure 4 In the software structure shown, the headphones can detect sensor data and audio data using sensor module 41 and audio module 43. Then, sensor feature extraction module 42 and audio feature extraction module 44 extract sensor features and audio features from the sensor data and audio data, respectively. Next, feature fusion module 45 concatenates the sensor features and audio features to generate a fused feature. Finally, fusion feature decision module 46 determines whether the headphones are being worn based on the fused feature. Thus, once the headphones are being worn, the corresponding task is initiated. In summary, this improves the accuracy of headphone wear detection, enhances user experience, and avoids increasing headphone power consumption.

[0110] The following is a combination of the above. Figure 1 The application scenario shown will use wireless headphones as an example to illustrate the wearing detection method provided in the embodiments of this application. Figure 5 This is a schematic flowchart illustrating a wear detection method according to an embodiment of this application, as shown below. Figure 5 As shown, the method may include the following steps S501-S506.

[0111] S501, the earphone responds to the first trigger operation and collects sensor data and audio data.

[0112] In practical applications, users typically perform a wearing operation to use the headphones. For example, a user would pick up the headphones and move them continuously for a period of time until they are worn on the ears at an angle. In some embodiments of this application, a first trigger operation is used to instruct the headphones to initiate wear detection. For example, the first trigger operation may include a user's touch operation on the headphones, such as pinching the headphones; another example is a user's action of separating the headphones from the headphone case, such as taking the headphones out of the headphone case; yet another example is a user's action of touching and moving the headphones, such as taking the headphones out of the headphone case. The first trigger operation can be regarded as an activation signal for the headphones to perform wear detection, thereby the headphones begin wear detection after the first trigger operation to determine whether the headphones are in a wearing state.

[0113] For example, when the earphones are stored in the earphone case and the case is opened, the capacitance signal strength collected by the capacitive sensor in sensor module 41 for touch detection is intensity value 1. After the user touches the earphones, the capacitance signal strength collected by the capacitive sensor will increase significantly, such as increasing to intensity value 2. Therefore, the earphones can determine that the user has touched the earphones based on the change in capacitance signal strength. At this time, the earphones determine that the first trigger operation has been detected, and thus begin wearing detection.

[0114] For example, a proximity sensor can be installed at the bottom of the earphone stem 123 or inside the earphone case. The earphone can detect separation from the earphone case using the proximity sensor. For instance, when the proximity sensor detects sufficient reflected light, it indicates that there is an object near the earphone, meaning the earphone is stored in the earphone case. When the proximity sensor detects insufficient reflected light, it indicates that the earphone has separated from the earphone case, meaning the user has taken the earphone out. Therefore, the earphone can determine whether it has separated from the earphone case based on changes in reflected light. At this point, the earphone detects the first trigger operation and begins wearing detection. It should be noted that separation of the earphone from the earphone case can mean the earphone does not touch the earphone case, or the earphone is completely detached from the earphone case.

[0115] This application embodiment uses a user's touch operation on the earphones as an example for the first trigger operation. The first trigger operation can also be a user picking up the earphones. It is understood that picking up the earphones is equivalent to the user touching the earphones, and there is a certain distance between the earphones and the charging case. Of course, it can also be an operation by the user on a trigger button located on the charging case or the earphones. The button can include physical buttons and virtual buttons, etc. This application embodiment does not impose any particular limitation on the first trigger operation; it can be specifically designed according to the actual usage and functional settings of the wireless earphones.

[0116] As can be seen, by responding to the first trigger operation before collecting data, the earphones prevent the sensor module 41 and audio module 43 from being continuously operational. This reduces the power consumption of the wireless earphones, extends the battery life, and improves the user experience.

[0117] To provide a detailed explanation of real-world headphone usage scenarios, Figure 8 This is a schematic diagram illustrating a scenario where a user is wearing headphones, as shown in an embodiment of this application. Figure 8 As shown in the illustration, this scenario includes a user touching the headphones (see...). Figure 8 (A) The user picks up the headphones and moves them continuously for a period of time (see also...) Figure 8 (B) and the user wearing the headphones in their ears (see also...) Figure 8The process involves three steps (C). Specifically, from the perspective of the earphone, these three steps include the process of the earphone moving to the ear (1) and the process of the earphone touching the ear (2).

[0118] For example, such as Figure 8 As shown in (A), the data collected by sensor module 41 and audio module 43 differs before and after the user touches the headphones. For example, before the user touches the headphones, the capacitance signal value collected by the capacitance sensor in sensor module 41 is 0. After the user touches the headphones, the capacitance signal value collected by the capacitance sensor is 100. Similarly, before the user touches the headphones, the audio energy collected by audio module 43 is low. After the user touches the headphones, the audio energy collected by audio module 43 is increased. Therefore, after the user touches the headphones, the headphones detect the user's touch operation and initiate the wearing detection process.

[0119] In this embodiment, after receiving the first trigger operation, the earphone controls the internal sensor module 41 to collect sensor data and controls the audio module 43 to collect audio data. (Continue to see...) Figure 8 The sensor data and audio data differ for each process. For ease of description, the sensor data during process 1 (when the earphone moves to the ear) can be referred to as the first target data, and the sensor data during process 2 (when the earphone touches the ear) can be referred to as the second target data. Similarly, the audio data during process 1 (when the earphone moves to the ear) can be referred to as the third target data, and the audio data during process 2 (when the earphone touches the ear) can be referred to as the fourth target data.

[0120] In some embodiments, the sensor data for the earphone moving to the ear is the sensor data corresponding to the earphone not being worn in the ear. The sensor data for the earphone touching the ear is the sensor data corresponding to the earphone being worn in the ear. The sensor data corresponding to different states are different. The earphone can determine the first target data and the second target data based on the changes in the collected sensor data. It is understood that the first target data may include the data collected by the sensor when the earphone is not worn in the ear, which can be used to characterize the contact area of ​​the earphone cap 124 in contact with the user. The second target data may include the data collected by the sensor when the earphone is worn in the ear, which can be used to characterize the contact area of ​​the earphone cap 124 in contact with the user. During the process of wearing the earphone, the contact area of ​​the earphone cap 124 in contact with the user is smaller than the contact area of ​​the earphone cap 124 in contact with the user. For example, the contact area of ​​the earphone cap 124 and the user represented by the sensor signal of the second target data is significantly larger than the contact area of ​​the earphone cap 124 and the user represented by the sensor signal of the first target data, and the difference between the areas is greater than a preset value of 1. For example, the sensor signal representing the second target data indicates that the contact area between the earphone cap 124 and the human ear is greater than or equal to a preset value 2, while the sensor signal representing the first target data indicates that the contact area between the earphone cap 124 and the human ear is less than the preset value 2. Therefore, if the contact area represented by the sensor data significantly increases or changes from less than the preset value 2 to greater than or equal to the preset value 2, the headphones can determine that the collected sensor data includes both the first and second target data. Simultaneously, it can also determine that the headphones have switched from an unworn in-ear state to a worn in-ear state. It is understandable that if the contact area represented by the sensor data does not significantly change or is less than the preset value 2, the headphones can determine that the collected sensor data includes the first target data, and simultaneously determine that the headphones are still in an unworn in-ear state.

[0121] In some embodiments, the audio data when the headphones move to the ear includes audio data corresponding to the open field when the headphones are not worn in the ear. The audio data when the headphones touch the ear includes audio data corresponding to the closed field when the headphones are worn in the ear. The audio data corresponding to different states are different. The open field can also be called the headphone-to-air state, where the headphone's speaker and microphone are exposed to the air. Therefore, in the open field, most of the audio signal dissipates into the air, and the microphone picks up a relatively small amount of audio signal energy. The closed field can also be called the headphone-closed state, where the headphones are worn on the ear (headphone cap 124 is inserted into the ear canal). In this state, the headphone's speaker and microphone face the ear canal, forming a closed space. Therefore, the audio signal energy dissipation in the closed field is less than that in the open field, and the microphone picks up a larger amount of audio signal energy than in the open field.

[0122] Therefore, the headphones can determine the third and fourth target data based on changes in the collected audio data, and also determine whether a switch has occurred between a closed field and an open field. The third target data includes audio data corresponding to the open field, representing the audio signal in that field, while the fourth target data includes audio data corresponding to the closed field, representing the audio signal in that field. The signal energy of the audio signal in the closed field is greater than that in the open field. For example, if the signal energy of the audio signal corresponding to the fourth target data is greater than that of the audio signal corresponding to the third target data, and the difference in energy is greater than a preset value of 3, then it can be determined that the headphones have switched from an open field to a closed field. Similarly, if the signal energy of the audio signal corresponding to the fourth target data is greater than or equal to a preset value of 4, while the signal energy of the audio signal corresponding to the third target data is less than a preset value of 4, it can also be determined that the headphones have switched from an open field to a closed field. Therefore, if the signal energy of the audio signal represented by the audio data significantly increases or changes from less than a preset value of 4 to greater than or equal to a preset value of 4, the headphones can determine that the collected audio data includes both the third and fourth target data. Simultaneously, it can also determine whether the headphones have switched from an unworn in-ear state to a worn in-ear state. Understandably, if the signal energy of the audio signal represented by the audio data does not change significantly or is less than a preset value of 4, the headphones can determine that the collected audio data includes the third target data, and can also determine that the headphones are still in an unworn in-ear state.

[0123] As can be seen, the embodiments of this application can accurately determine whether the headphones are currently being worn by collecting sensor data and audio data in the above two processes. For example, if the difference between the contact area between the headphone cap 124 and the human ear in the second target data and the contact area between the headphone cap 124 and the human ear in the first target data is greater than a preset value of 1, and the difference between the signal energy of the fourth target data and the signal energy of the third target data is greater than a preset value of 3, then it is determined that the headphones are currently being worn.

[0124] In one possible implementation, after receiving a first trigger operation, the headphones control the sensor path to activate the sensors in sensor module 41 for data acquisition. These sensors may include one or more of a gravity sensor, proximity sensor, capacitance sensor, temperature sensor, and touch sensor. Simultaneously, the headphones control the audio path to activate the speaker and microphone in audio module 43. The speaker plays a preset audio signal, which may be a single-frequency, narrowband infrasound or ultrasonic signal. The microphone collects the preset audio signal played by the speaker, which may include preset infrasound, ultrasonic, and low-frequency audio signals. During the transition from an open field to a closed field, the microphone can also collect audio data corresponding to the wearing action. This audio data may include a first audio signal corresponding to the change in air pressure caused by the headphones rubbing against the ear canal, and a second audio signal corresponding to the same action. It is understood that the audio data of the headphones touching the ear also includes audio data corresponding to the wearing action, allowing the extraction of friction and impact sound characteristics from the collected first and second audio signals.

[0125] When a user touches the earphones, the earphones detect the first trigger operation. This means that as the earphones move towards the user's ear, the speaker begins playing a preset audio signal, and the microphone simultaneously picks up this preset audio signal. As the earphones touch the user's ear, the friction between the earphones and the ear canal also generates a corresponding audio signal. At this point, the microphone picks up both the preset audio signal and the aforementioned first and second audio signals.

[0126] Because the signal energy of the preset audio signal collected by the microphone in a closed environment (when the earphone touches the ear) is greater than the signal energy of the preset audio signal collected by the microphone in an open environment (when the earphone moves to the ear), it can be understood that the signal energy of the preset audio signal corresponding to the wireless earphone in the wearing state is greater than the signal energy of the preset audio signal corresponding to the earphone in the non-wearing state. The non-wearing state includes not only scenarios where the earphone is moved to the ear, but also scenarios such as the earphone being placed on a table or being handled / played with.

[0127] Simultaneously, during the process of the earphone touching the ear, the microphone collects audio data not only from the enclosed field but also from the audio data corresponding to the aforementioned wearing and insertion action. Therefore, the signal energy of the audio signal collected by the microphone during the earphone touching the ear is greater than the signal energy of the audio signal collected during the earphone moving towards the ear. Thus, the switch from an open field to an enclosed field can be determined based on two factors: the increase in audio signal energy and whether the audio data collected by the microphone includes both the first and second audio signals.

[0128] Furthermore, the switching process between open and closed fields can be determined by the characteristics of friction and impact sounds generated during the wearing action. For example, based on specific thresholds, the first and second audio signals collected by the microphone can be evaluated. When the signal characteristic values ​​corresponding to the first and second audio signals are greater than the specific threshold, it can be determined that the headphones have switched from an open field to a closed field. For example, when the time-domain clipping characteristic of the first audio signal and the Mel-frequency cepstral coefficients of the mid-low frequency band of the second audio signal are both greater than specific thresholds, it can be determined that the headphones have switched from an open field to a closed field. As another example, when the sum of the time-domain clipping characteristic value of the first audio signal and the Mel-frequency cepstral coefficients of the mid-low frequency band of the second audio signal is greater than a specific threshold, it can be determined that the headphones have switched from an open field to a closed field. Therefore, it can be determined whether the headphones have switched from an unworn in-ear state to a worn in-ear state.

[0129] In one possible implementation, during the subsequent process of determining whether the headphones are being worn based on sensor features and audio features, the headphones can also determine whether they are being worn based on sensor features and target audio features. That is, if the target audio features meet certain constraints, the determination of whether the headphones are being worn can be made further by combining sensor features. For example, the time-domain clipping feature of the first audio signal represented by the target audio features can be greater than a first preset threshold, and the Mel-frequency cepstral coefficient of the second audio signal in the mid-to-low frequency band can be greater than a second preset threshold. Similarly, the signal energy of the preset audio signal represented by the target audio features can also be greater than a third preset threshold. Therefore, this embodiment of the application, by acquiring sensor data and audio data in real time, facilitates accurate subsequent determination of whether the headphones are being worn, avoiding errors in the determination process.

[0130] S502, the earphone extracts sensor features from sensor data and audio features from audio data.

[0131] In one possible implementation, the earphone can use an internal sensing feature extraction module 42 to extract features from the collected sensor data to obtain sensing features corresponding to the earphone moving to the ear and the earphone touching the ear.

[0132] Simultaneously, the headphones can use their internal audio feature extraction module 44 to extract features from the collected audio data, obtaining the audio features corresponding to the headphones moving to the ear and the headphones touching the ear. It can be understood that the audio features of the headphones moving to the ear include the audio features corresponding to the open field when the headphones are not worn in the ear. The audio features of the headphones touching the ear include the audio features corresponding to the closed field when the headphones are worn in the ear.

[0133] For example, headphones can filter and transform the waveform corresponding to the acquired capacitance signal to obtain sensing characteristics such as capacitance value and amplitude. Simultaneously, headphones can also process the waveform corresponding to the acquired audio signal to obtain audio characteristics such as waveform clipping characteristics, signal envelope characteristics, frequency energy characteristics of the frequency domain signal, and Mel-frequency spectrum values ​​corresponding to low-frequency or mid-frequency bands.

[0134] In one possible implementation, during the feature extraction process of the audio data by the audio feature extraction module 44, since the characteristics of the audio data are difficult to reflect in the time domain, the audio feature extraction module 44 can use a Fast Fourier Transform (FFT) to transform the audio data into the frequency domain, so that audio features can be obtained subsequently based on the energy distribution in the frequency domain. Next, the audio feature extraction module 44 converts the spectrum obtained after the FFT transform into the energy spectrum of the audio data. Finally, based on the energy distribution in the energy spectrum, the audio features corresponding to the earphone moving to the ear and the earphone touching the ear are extracted.

[0135] In another possible implementation, the above feature extraction process can be implemented using multiple feature extraction models. For example, the headphones input sensor data into a first feature extraction model, which performs convolution processing through multiple convolutional layers to obtain sensor features. Similarly, the headphones input audio data into a second feature extraction model, which performs convolution processing through multiple convolutional layers to obtain audio features. The first and second feature extraction models can be convolutional neural network models, support vector machine network models, or decision tree network models, etc.

[0136] Furthermore, the feature extraction model is trained before it is used to extract features. During training, a large amount of sensor data and audio data can be used as training samples. Additionally, large amounts of data from the first target, second target, third target, and fourth target can be used as training samples to enable the feature extraction model to learn the ability to extract sensor and audio features.

[0137] In some embodiments, the aforementioned sensor feature extraction module 42, audio feature extraction module 44, or feature extraction model can be located not only on the earphone side but also on the cloud-side server side. This reduces the earphone's own resource utilization and lowers its operating costs. It also saves time for the earphone to process data. For example, if the server side is configured, the audio module 43 and sensor module 41 can send the collected sensor data and audio data to the server side, allowing the server side to perform feature extraction using the sensor feature extraction module 42, audio feature extraction module 44, or feature extraction model to obtain the extraction results. Finally, the extracted results are returned to the earphone side.

[0138] As can be seen, the embodiments of this application employ two separate functional modules or feature extraction models to extract features from sensor data and audio data. This helps to remove redundant information from the data and enables the rapid and accurate acquisition of sensor and audio features. This improves the efficiency and accuracy of obtaining sensor and audio features, and also facilitates subsequent feature fusion. Furthermore, it provides a better ability to distinguish between user-wearing headphone scenarios and other interference scenarios, thus improving the accuracy of headphone wearing detection.

[0139] Then, the headphones can determine the wearing status based on the sensing features and audio features. In some embodiments, as shown in steps S503-S504, the headphones can specifically determine the wearing status based on the fused features of the sensing features and audio features.

[0140] The S503 headphones fuse sensor features and audio features to obtain fused features.

[0141] In one possible implementation, the headphones can use an internal feature fusion module 45 to fuse the extracted sensor features and audio features to obtain fused features.

[0142] For example, the headphones can construct a two-dimensional planar region from the obtained sensing features and audio features to obtain fused features. For instance, the sensing feature is S, and the audio feature is A. Thus, the two-dimensional coordinate point (S, A) can represent the fused feature. The two-dimensional planar region is composed of the value ranges of the sensing feature and the audio feature. For another example, suppose the range of sensing feature S is [20, 500], and the range of audio feature A is [10, 1000]. The currently obtained sensing feature has a feature value of 400, and the audio feature has a feature value of 800. Then, the two-dimensional coordinate point (400, 800) can represent the fused feature.

[0143] In one possible implementation, the headphones can also directly concatenate the aforementioned sensing features and audio features to obtain fused features. This application does not specifically limit the particular feature fusion method.

[0144] As can be seen, this embodiment of the application increases the richness of features by fusing sensor features and audio features, while simplifying operation and saving data processing time. This avoids long waiting times for users and improves the user experience.

[0145] In another possible implementation, during the process of fusing sensor features and audio features, the feature fusion module 45 can also set weights for the fused sensor features and audio features according to preset rules, and then fuse the weighted sensor features and audio features to obtain fused features.

[0146] For example, in the process of setting weights for sensor features and audio features as described above, the priorities of sensor data and audio data can be pre-defined. For instance, the priority of sensor data can be higher than that of audio data, and the priority is positively correlated with the weight of the feature. Thus, the weight of the sensor feature corresponding to the sensor data is higher than that of the audio feature corresponding to the audio data. Conversely, for example, the priority of sensor data can be lower than that of audio data. Then, the weight of the sensor feature corresponding to the sensor data is lower than that of the audio feature corresponding to the audio data. In this way, the feature fusion module 45, by concatenating sensor features and audio features based on weights, can eliminate redundant information in the sensor features and audio features, making the obtained fused features clearer, thereby improving the accuracy of subsequent feature fusion decisions.

[0147] During feature fusion, the headphones can still use a feature fusion model to fuse audio features and sensor features. The specific implementation is similar to the feature extraction model described above, and will not be repeated here.

[0148] S504. Based on the fusion feature, the earphone determines whether it is currently being worn. If the earphone is currently being worn, proceed to S506; otherwise, proceed to S505.

[0149] In one possible implementation, the headphones can use an internal fusion feature decision module 46 to determine whether the headphones are currently being worn.

[0150] For example, the headphones compare the fusion feature value corresponding to the fusion feature with a second preset range. The second preset range includes the lowest to highest feature value corresponding to the headphones being worn. When the fusion feature value is within the second preset range, it indicates that the audio feature in the fusion feature shows that the headphones are currently in a closed environment, and the headphones may have been inserted into the ear; and the sensing feature in the fusion feature shows that the contact area between the headphone cap 124 and the ear is greater than or equal to a preset value of 2, indicating a relatively large contact area, suggesting the headphones may have been inserted into the ear. Therefore, it can be determined that the headphones are currently being worn. Similarly, when the fusion feature value is outside the second preset range, it indicates that the audio feature in the fusion feature shows that the headphones are currently in an open environment, and the headphones may not have been inserted into the ear; and the sensing feature in the fusion feature shows that the contact area between the headphone cap 124 and the ear is less than a preset value of 2, indicating a relatively small contact area, suggesting the headphones may not have been inserted into the ear. Therefore, it can be determined that the headphones are currently not being worn.

[0151] For example, headphones can numerically divide the two-dimensional planar region corresponding to the fusion feature and determine the wearing state based on the values ​​of different regions. Continuing with the example of a sensor feature with a feature value of 400 and an audio feature with a feature value of 800, the two-dimensional coordinate point (400, 800) represents the fusion feature. Then, assuming the determination rule is [S>200, A>500], the current wearing state can be determined based on the fusion feature (400, 800). For another example, the above determination rule can also simultaneously satisfy the following two conditions: the first condition is: 2×A+0.3×S>300; the second condition is: A>700. This application does not specifically limit the determination conditions; they can be set according to actual circumstances.

[0152] Thus, if the position of the two-dimensional plane region containing the two-dimensional coordinate point corresponding to the fused feature meets the determination condition, it can be determined that the current headphones are being worn. Similarly, if the position of the two-dimensional plane region containing the two-dimensional coordinate point corresponding to the fused feature does not meet the determination condition, it can be determined that the current headphones are not being worn.

[0153] Since different fusion features correspond to different fusion feature values, it is necessary to set the difference features between the wearing and non-wearing states of the headphones according to the specific fusion feature values. Therefore, different preset ranges and judgment rules can be set. At the same time, during the process of the headphones making a judgment on the fusion features, the fusion feature judgment model can still be used to judge the fusion features. The specific implementation method is the same as the feature extraction model mentioned above, and will not be repeated here.

[0154] Related technologies determine wearing status based on sensor data. For example, if the sensor data at a specific location in the headphones changes significantly, it can be determined that the headphones are being worn. In scenarios such as when a user plays with the headphones or covers them, the user can easily touch the sensor at a specific location in the headphones. Consequently, the sensor data detected at that specific location will change significantly, leading to a misjudgment that the headphones are currently being worn. This application, however, fuses the sensor features corresponding to the sensor data and the audio features corresponding to the audio data. Based on the feature changes corresponding to the fused features, it can accurately determine whether the headphones are being worn in scenarios such as when a user plays with the headphones or covers them. For example, when the fused feature value is within a second preset range, it indicates that the audio features in the fused features show that the headphones are in a closed environment, and the sensor features in the fused features indicate that the contact area between the headphone cap 124 and the ear is greater than or equal to a preset value 2. Therefore, it can be determined that the headphones have moved into the ear canal, thus accurately determining that the headphones are currently being worn. Therefore, in scenarios such as users playing with or covering the headphones, even if the sensor data changes significantly due to the user touching the sensor with their finger, the aforementioned audio features will not be detected, nor will the fused feature value within the aforementioned second preset range be obtained, thus failing to determine whether the headphones are currently being worn. This application embodiment can more accurately determine whether headphones are being worn by combining audio data, avoiding the situation where headphones are mistakenly turned on based on a single sensor's data.

[0155] Furthermore, if headphones are exposed to temperature fluctuations or water, they are susceptible to interference from external temperature, humidity, electromagnetic interference, and sensor conditioning circuitry. This can cause abnormalities in the sensor signals, leading to inaccurate headphone wear detection results. This embodiment combines sensor characteristics with audio characteristics for wear detection. Even if the sensor signal is abnormal, the audio characteristics can indicate that the headphones are in a closed environment, thus determining that the headphones are being worn. This avoids situations where abnormal sensor signals cause the headphones to erroneously activate their functions.

[0156] In some embodiments, considering actual user scenarios, the headphones in this application embodiment may collect sensor data and audio data only once after a trigger operation, and determine whether the headphones are being worn based on the sensor data and audio data. Of course, the headphones may also collect sensor data and audio data multiple times at periodic time intervals, and determine whether the headphones are being worn multiple times based on the sensor data and audio data. The number of times the sensor data and audio data are collected and the subsequent determination of whether the headphones are being worn can be determined according to constraint rules. These constraint rules may include the total power consumption time of the headphones and the output time of the wear detection result, etc. This application embodiment does not specifically limit the number of times the headphones perform the above-mentioned wear detection process.

[0157] S505, the headphones do not start working tasks.

[0158] S506, Headphones start working task.

[0159] In some embodiments, if the headphones are not currently being worn, the headphones will not initiate any tasks. If the headphones are currently being worn, the headphones will initiate any tasks. It is understood that when the headphones are being worn, it is confirmed that the user has placed the headphones in their ears, and the headphones will perform the corresponding tasks. These tasks may specifically include audio playback, message notifications, voice calls, and singing / voiceover activities. When the headphones are not being worn, it indicates that the user is not currently placing the headphones in their ears, and the headphones will not initiate any corresponding tasks.

[0160] In other embodiments, after the headphones initiate a task, the user typically removes the headphones from their ears and places them on a static object or in their case to stop using them. If the headphones determine that they are not currently being worn, they can terminate the task. For example, when the user removes the headphones from their ears, the headphones detect a second trigger action by the user and perform a disengagement detection. Finally, based on the detection result, the headphones determine that they are not currently being worn and terminate the task.

[0161] In one implementation, after detecting the second trigger operation, the earphone controls the internal sensor module 41 to collect sensor data and the audio module 43 to collect audio data. Since the sensor data and audio data differ under different states, the earphone can determine whether it is in a non-wearing state based on changes in the combined sensor and audio data. For example, sensor data can characterize the contact area between the ear and the earphone cap 124. If the earphone detects a significant decrease in the contact area between the earphone and the human body, or a change from greater than or equal to a preset value 2 to less than a preset value 2, it can be determined that the earphone has switched from an in-ear wearing state to a non-in-ear wearing state. Audio data can characterize audio signals in closed and open fields. If the earphone detects a significant decrease in the signal energy of the audio signal, or a change from greater than or equal to a preset value 4 to less than a preset value 4, it can be determined that the earphone has switched from a closed field to an open field, and thus, the earphone has switched from an in-ear wearing state to a non-in-ear wearing state. Therefore, when the headphones detect a significant decrease in the contact area between the headphones and the human body and a significant decrease in the signal energy of the audio signal, or when the contact area between the headphones and the human body changes from greater than or equal to a preset value 2 to less than a preset value 2 and the signal energy of the audio signal changes from greater than or equal to a preset value 4 to less than a preset value 4, the headphones can be determined to be in a non-wearing state.

[0162] For example, the headphones can also fuse the sensing features corresponding to the sensor data and the audio data corresponding to the audio data, and compare the fused feature value with a second preset range. The second preset range includes the lowest to highest feature value corresponding to the headphones not being worn. Therefore, if the headphones detect that the fused feature value is within the second preset range, it indicates that the audio features in the fused features suggest the headphones are in an open environment and may not be inserted into the ear; and the sensing features in the fused features indicate that the contact area between the headphone cap 124 and the ear is less than a preset value of 2, suggesting the headphones may not be inserted into the ear. Thus, it can be determined that the headphones are currently not being worn.

[0163] As can be seen, the embodiments of this application can jointly determine the current wearing status of the headphones based on the fused features, combining both sensing and audio features, thereby providing users with more accurate wearing detection results. This reduces the probability of accidental triggering by users and improves the accuracy and reliability of headphone wearing detection. Furthermore, the embodiments of this application provide multiple functional modules to implement the wearing detection method, which can improve performance in terms of cost, size, and computational accuracy, enhancing the adaptability of headphone wearing detection and providing users with a better user experience.

[0164] The following continues in conjunction with the above. Figure 1The application scenario shown, still using headphones as an example, will be used to illustrate the wearing detection method provided in the embodiments of this application. Figure 6 As shown, in another embodiment, the above steps S501-S502 can be replaced by steps S601-S604.

[0165] S601, the earphone responds to the first trigger operation and collects sensor data.

[0166] S602, the sensing characteristics of the earphone extract sensor data.

[0167] In one implementation, after detecting the first trigger operation, the earphone controls the sensor path to open the sensor module 41. It then controls the sensors in the sensor module 41 to collect sensor data. Next, the earphone controls the sensor feature extraction module 42 to extract features from the collected sensor data to obtain sensor features. These sensor features include those corresponding to the earphone moving to the ear and the earphone touching the ear. The specific process of collecting sensor data and extracting sensor features is similar to that in the first embodiment described above, and will not be repeated here.

[0168] S603: The headphones determine whether to collect audio data based on sensor characteristics. If audio data is collected, S604 is executed; otherwise, S607 is executed.

[0169] S604: The earphone collects audio data and extracts the audio features of the audio data.

[0170] In one implementation, the headphones determine whether to trigger the audio module 43 to collect audio data based on the extracted sensor features. For example, the headphones can collect audio data when a preset condition is met based on the sensor features. The preset condition includes the feature value corresponding to the sensor feature being within a first preset range. Thus, when the feature value corresponding to the sensor feature is within the first preset range, it can be preliminarily determined that the headphones are in the process of being moved towards the ear. Therefore, the headphones can perform an initial wearing determination based on the extracted sensor features to obtain an initial determination result. This initial determination result is used to characterize whether the headphones are currently being moved towards the ear.

[0171] For example, the sensing characteristic is the capacitance signal value determined based on sensing data, with a capacitance signal threshold set to 50. It can be understood that the process of the earphone moving towards the ear is the process of the earphone approaching the ear. That is, as the earphone gets closer to the ear, the capacitance signal value of the capacitance sensor will increase significantly. Therefore, when the capacitance signal value is greater than or equal to 50, it indicates that the earphone is approaching the ear, and it is initially determined that the earphone is in the process of moving towards the ear. Then, after determining that the wireless earphone is in the process of moving towards the ear, it triggers the subsequent step of collecting audio data.

[0172] Therefore, if the initial determination indicates that the earphone is close to the ear, the earphone controls the audio module 43 to collect audio data and extract audio features from the audio data. This allows for a more accurate final determination of whether the earphone is being worn, based on the fusion of audio and sensor features. The specific process of collecting audio data and extracting audio features is similar to that in the first embodiment described above, and will not be repeated here.

[0173] In this embodiment of the application, steps S605-S608 are... Figure 5 Steps S503-S506 in the method shown are similar and will not be repeated here.

[0174] As can be seen, compared to the previous embodiment, the difference in this embodiment is that the headphones can perform an initial wearing determination based on the extracted sensing features to obtain an initial determination result. Then, based on the initial determination result, it is determined whether to trigger the audio module 43 to collect audio data. Thus, if the initial determination result indicates that the headphones are not currently close to the ear, the headphones do not need to trigger the audio module 43 to operate, which reduces the power consumption of the headphones and increases their battery life. Simultaneously, wearing detection based on fused features improves the reliability of the headphones during wearing detection.

[0175] The following continues in conjunction with the above. Figure 1 The application scenario shown uses wireless headphones as an example to illustrate the wearing detection method provided in this application embodiment. Figure 7 As shown, step S601 can be replaced by steps S701-702.

[0176] S701, the earphone responds to the first trigger operation and collects sensor data.

[0177] S702. When the duration of the earphone collecting sensor data is greater than or equal to the preset duration, the earphone controls the microphone to power on.

[0178] The purpose of powering on the microphone via the headphones is to pre-power on the microphone, allowing it to operate for a certain period beforehand. This ensures that when the headphones subsequently collect audio data based on sensor features, the microphone is already in a stable state, improving the accuracy of the acquired audio data. Furthermore, the audio data acquired by the microphone in a stable state can be used for feature extraction, enabling wear detection. This reduces latency in the wear detection process and improves the user experience.

[0179] Typically, headphones can include multiple microphones, such as a main microphone and other microphones. The main microphone is used to pick up the user's audio data, while the other microphones provide data input for algorithms such as Acoustic Echo Cancelling (AEC), Automatic Gain Control (AGC), and Active Noise Control (ANC) (the 3A algorithms). Therefore, in one implementation, the headphones can pre-power on the main microphone and then power on the other microphones when extracting features from the audio data. This reduces the headphone's power consumption and increases battery life. Alternatively, the headphones can power on the main microphone and other microphones simultaneously to ensure smoother operation during feature extraction, improving the accuracy of the acquired audio data. This application does not specifically limit which microphones the headphones power on or the order in which they are powered on.

[0180] In one possible implementation, for example, the preset duration can be 200ms. Then, after the headphone control sensor module 41 has collected sensor data for a duration greater than or equal to 200ms, the headphone powers on the microphone, which is then used for subsequent audio data collection. After the headphone controls the microphone to power on, and the headphone control sensor module 41 has collected sensor data for a duration greater than or equal to the preset duration, the headphone can then perform the subsequent step of extracting sensor features from the sensor data. For example, the preset duration can be 400ms. Then, after the headphone turns on the microphone, the headphone control sensor module 41 collects sensor data for a duration greater than or equal to 400ms. Next, the headphone begins extracting sensor features from the sensor data to facilitate subsequent determination of whether to continue collecting audio data based on these features. The specific process of collecting sensor data and extracting sensor features is similar to that in the above embodiments and will not be repeated here. This application embodiment only uses preset durations of 200ms and 400ms as examples; the preset duration can be designed according to actual conditions.

[0181] In this embodiment of the application, steps S703-S705 are... Figure 6 Steps S602-S604 in the method shown are similar. Steps S706-S709 are the same. Figure 5 Steps S503-S506 in the method shown are similar and will not be repeated here.

[0182] As can be seen, compared to the previous embodiment, the difference in this embodiment is that the microphone is pre-activated during the acquisition of sensor data by the earphone. Then, based on the extracted sensor features, it is determined whether to trigger the audio module 43 to acquire audio data. In this way, by pre-activating the microphone, stable operation of the microphone during audio acquisition is ensured, reducing the latency of the wear detection process. Simultaneously, pre-determining based on sensor features eliminates the need for the audio module 43 and sensor module 41 to work synchronously, reducing the earphone's power consumption.

[0183] It is understood that, in order to achieve the above functions, the headphones include hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0184] This embodiment can divide the sensor module and audio module into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. The module division in this embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0185] This application also provides a wear detection device, such as... Figure 9 As shown, the wear detection device 900 may include an acquisition module 910 and a processing module 920. The acquisition module 910 acquires sensor data and audio data, while the processing module 920 extracts the corresponding sensor features and audio features from the sensor data and audio data, and then concatenates the extracted sensor features and audio features to obtain a fused feature. The processing module 920 further determines the current state of the earphone based on the fused feature, thus obtaining a wear detection result.

[0186] Optionally, in some embodiments, the method performed by the acquisition module 910 can be performed by multiple modules. For example, the wear detection device 800 includes a sensor module and an audio module, wherein the sensor module is used to acquire sensor data and the audio module is used to acquire audio data. As another example, the wear detection device 800 further includes a sensor feature extraction module, an audio feature extraction module, a feature fusion module, and a fusion feature decision module. The sensor feature extraction module is used to extract sensor features from the sensor data. The audio feature extraction module is used to extract audio features from the audio data. The feature fusion module is used to fuse the sensor features and the audio features to obtain fused features. The fusion feature decision module is used to obtain the wear detection result based on the fused features.

[0187] The names of the modules mentioned above are merely illustrative examples. Each module can execute its corresponding method, and this application does not limit this.

[0188] This application also provides a wearable device, such as the aforementioned wireless headphones, etc. Figure 10 As shown, the wearable device 1000 may include one or more processors 1010, memory 1020 and communication interfaces 1030.

[0189] The memory 1020, communication interface 1030, and processor 1010 are coupled together. For example, the memory 1020, communication interface 1030, and processor 1010 can be coupled together via bus 1040.

[0190] The communication interface 1030 is used for data transmission with other devices. The memory 1020 stores computer program code. The computer program code includes computer instructions, which, when executed by the processor 1010, cause the wearable device to perform the wear detection method described in this embodiment.

[0191] The processor 1010 may be a processor or controller, such as a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. The processor may also be a combination that implements computational functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0192] The bus 1040 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1040 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0193] This application also provides a computer-readable storage medium storing computer program code. When the processor executes the computer program code, the electronic device executes the relevant method steps in the above method embodiments.

[0194] This application also provides a computer program product that, when run on a computer, causes the computer to execute the relevant method steps described in the above method embodiments.

[0195] The wearable detection device, wearable device, computer storage medium or computer program product provided in this application are all used to perform the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0199] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0200] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0201] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting wearability, characterized in that, Applied to wireless earphones, the wireless earphones including an earphone body and an earphone case, the method includes: In response to a trigger operation, the sensor features corresponding to the sensor data and the audio features corresponding to the audio data are acquired; the trigger operation includes a user touch operation on the earphone body, a user separation operation on the earphone body from the earphone case, or a user touch and move operation on the earphone body. The wireless earphone is determined to be in a wearing state based on the sensing features and target audio features. The time-domain clipping feature of the first audio signal represented by the target audio features is greater than a first preset threshold, and the Mel-frequency cepstral coefficient of the second audio signal in the mid-low frequency band is greater than a second preset threshold. The first audio signal is the audio signal corresponding to the change in air pressure caused by the wireless earphone touching the human ear in the audio data, and the second audio signal is the audio signal corresponding to the friction generated when the wireless earphone touches the human ear in the audio data.

2. The method according to claim 1, characterized in that, The acquisition of the sensing features corresponding to the sensor data and the audio features corresponding to the audio data includes: The sensor data and the audio data are collected simultaneously. The sensor data and the audio data are subjected to feature extraction to obtain the sensing features and the audio features.

3. The method according to claim 1, characterized in that, The acquisition of the sensing features corresponding to the sensor data and the audio features corresponding to the audio data includes: Collect the sensor data and extract features from the sensor data to obtain the sensing features; When the preset conditions are met based on the sensing features, the audio data is collected; The audio data is subjected to feature extraction to obtain the audio features.

4. The method according to claim 3, characterized in that, The preset conditions include the feature value corresponding to the sensing feature being within a first preset range; when the feature value corresponding to the sensing feature is within the first preset range, the wireless earphone is in the process of moving to the human body.

5. The method according to claim 3 or 4, characterized in that, The method further includes: When the acquisition time of the sensor data is greater than or equal to the preset time, the microphone is powered on. The acquisition of the audio data includes: The audio data is acquired using the powered-on microphone.

6. The method according to claim 1, characterized in that, The sensor data and the audio data include signals from the movement of the wireless earphones and their contact with a human body; the audio data includes preset audio signals played by the speaker and / or audio signals generated during the contact between the earphones and a human body.

7. The method according to claim 6, characterized in that, The audio signal generated when the earphone body touches the human body includes the first audio signal and the second audio signal.

8. The method according to claim 7, characterized in that, The signal energy of the preset audio signal corresponding to the wireless earphone when it is worn is greater than the signal energy of the preset audio signal corresponding to the wireless earphone when it is not worn, and the audio characteristics corresponding to different signal energies are different.

9. The method according to claim 8, characterized in that, The signal energy of the preset audio signal characterized by the target audio feature is greater than a third preset threshold.

10. The method according to claim 1, characterized in that, The method further includes: The sensing features and the audio features are fused to obtain fused features; The wireless earphone is determined to be in a wearing state based on the fusion features.

11. The method according to claim 10, characterized in that, The process of fusing the sensing features and the audio features to obtain fused features includes: The sensing features and the audio features are weighted accordingly; The weighted sensor features are concatenated with the audio features to obtain the fused features.

12. The method according to claim 11, characterized in that, Determining whether the wireless earphone is being worn based on the fusion features includes: The fusion feature value corresponding to the fusion feature is compared with the second preset range; If the fusion feature value is within the second preset range, then it is determined that the wireless earphone is in a wearing state; If the fusion feature value is outside the second preset range, then the wireless earphone is determined to be in a non-wearing state.

13. The method according to claim 12, characterized in that, When the fusion feature value is within the second preset range, the sensing feature in the fusion feature indicates that the contact area between the earphone cap of the earphone body and the human body is greater than the preset value, and the audio feature in the fusion feature indicates that the earphone body is in a closed field and the earphone body is inserted into the human ear.

14. The method according to claim 13, characterized in that, After determining whether the wireless earphone is being worn based on the fusion feature, the method further includes: When the wireless earphone is in the wearing state, control the wireless earphone to start the corresponding working task; When the wireless earphone is not being worn, control the wireless earphone to not start or terminate the corresponding working tasks.

15. The method according to claim 14, characterized in that, The sensor data includes posture data, motion data, distance data, and signal data during the movement of the wireless earphone and its contact with the human body; the sensor data is used to determine the proximity of the human body to the wireless earphone and / or the contact action between the human body and the wireless earphone.

16. A wireless earphone, characterized in that, The wireless earphone includes a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the wireless earphone to perform the wearing detection method as described in any one of claims 1-15.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, enable the computer to perform the wear detection method as described in any one of claims 1-15.

Citation Information

Patent Citations

  • Earphone wearing state detection method, device and headset

    CN109168102A

  • Wireless earphone control method and device, wireless earphone and storage medium

    CN110278509A

  • Earphone

    CN114979871A

  • Acoustic in ear detection for a hearable device

    US10491981B1