Earphone-based smart detection method and device, computer device and storage medium

Abstract

The application relates to an earphone-based intelligent detection method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining a first reflection signal of a first light signal; the first light signal is a light signal emitted by a red light source on a double-color light source substrate arranged in an earphone; performing earphone wearing detection based on the first reflection signal to obtain an earphone wearing detection result; in the case that it is determined that the earphone is in a wearing state based on the earphone wearing detection result, obtaining a second reflection signal of a second light signal; the second light signal is a light signal emitted by a green light source on the double-color light source substrate; performing heart rate detection based on the second reflection signal to obtain a heart rate detection result. The method can improve the utilization rate of the earphone.

Description

Technical Field

[0001] This application relates to the field of headphone technology, and in particular to a headphone-based intelligent detection method, device, computer equipment, and storage medium. Background Technology

[0002] With the rapid development of the consumer industry, various consumer electronics products are emerging in an endless stream, driving the development of all walks of life, and enabling the development and utilization of more and more new components. Listening to music is a way for people to relax and relieve stress during their leisure time, and choosing headphones to listen to music has become one of the most common choices for consumers. Therefore, headphones have become one of the mainstream consumer electronics products and are welcomed by consumers. However, at present, the function of headphones is limited to playing audio, resulting in low utilization of headphones. Summary of the Invention

[0003] Therefore, it is necessary to provide a headphone-based intelligent detection method, device, computer equipment, computer-readable storage medium, and computer program product that can improve headphone utilization by addressing the aforementioned technical problems.

[0004] In a first aspect, this application provides an intelligent detection method based on headphones, the method comprising:

[0005] Acquire a first reflected signal of the first optical signal; wherein, the first optical signal is an optical signal emitted by a red light source disposed on a dual-color light source substrate in the earphone;

[0006] Based on the first reflected signal, headphone wearing detection is performed to obtain headphone wearing detection results;

[0007] When it is determined that the headphones are in a wearing state based on the headphone wearing detection result, a second reflected signal of the second light signal is acquired; wherein, the second light signal is the light signal emitted by the green light source on the dual-color light source substrate;

[0008] Heart rate detection is performed based on the second reflected signal to obtain the heart rate detection result.

[0009] Secondly, this application also provides a smart detection device based on headphones, the device comprising:

[0010] The wearing detection module is used to acquire a first reflected signal of a first light signal; wherein, the first light signal is a light signal emitted by a red light source on a dual-color light source substrate disposed in the earphone; the earphone wearing detection is performed based on the first reflected signal to obtain an earphone wearing detection result;

[0011] The heart rate detection module is used to acquire a second reflected signal of a second light signal when the earphone is determined to be in a wearing state based on the earphone wearing detection result; wherein the second light signal is the light signal emitted by the green light source on the dual-color light source substrate; and to perform heart rate detection based on the second reflected signal to obtain a heart rate detection result.

[0012] Thirdly, this application also provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in any of the headphone-based intelligent detection methods provided in the embodiments of this application.

[0013] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the headphone-based intelligent detection methods provided in the embodiments of this application.

[0014] Fifthly, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the headphone-based intelligent detection methods provided in the embodiments of this application.

[0015] The aforementioned intelligent detection method, device, computer equipment, storage medium, and computer program product based on headphones can detect headphone wear by acquiring the first reflected signal of the first optical signal. When the headphone is determined to be worn based on the headphone wear detection result, it can detect heart rate by acquiring the second reflected signal of the second optical signal. Because it can perform both headphone wear detection and heart rate detection, it greatly enriches the functionality of headphones compared to traditional headphones that can only play audio, thus improving headphone utilization. Furthermore, since the red and green light sources are integrated on a dual-color light source substrate, compared to deploying separate red and green light sources, the additional space occupied by separate red and green light sources inside the headphone can be reduced, aligning with the trend of headphone miniaturization design. Attached Figure Description

[0016] Figure 1 This is an application environment diagram of an earphone-based intelligent detection method in one embodiment;

[0017] Figure 2 This is a flowchart illustrating an intelligent detection method based on headphones in one embodiment;

[0018] Figure 3 This is a schematic diagram illustrating the pulse transmission time interval in one embodiment;

[0019] Figure 4This is a schematic diagram of the time-sharing drive waveform in another embodiment;

[0020] Figure 5 This is a schematic diagram of the device layout in one embodiment;

[0021] Figure 6 This is a schematic diagram of the overall framework process in one embodiment;

[0022] Figure 7 This is a schematic diagram of the workflow in one embodiment;

[0023] Figure 8 This is a schematic diagram of an earphone-based intelligent detection device in one embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] The intelligent detection method based on headphones provided in this application can be applied to, for example... Figure 1 In the application environment shown, the earphone 102 is equipped with a dual-color light source substrate, a red light photosensitive sensor, a green light photosensitive sensor, and a time-division driving circuit. The time-division driving circuit drives the red and green light sources on the dual-color light source substrate to alternately emit light signals. The red light photosensitive sensor is used to collect the reflected red light and perform wearing detection on the earphone 102 based on the collected reflected red light to identify whether the earphone 102 is being worn. The green light photosensitive sensor is used to collect the reflected green light and perform heart rate detection based on the collected reflected green light.

[0026] It should be noted that the terms "first," "second," and similar terms used in this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Unless the context clearly indicates otherwise, the singular forms "a," "one," or "the," etc., do not indicate a quantity limitation, but rather indicate the presence of at least one. The quantities of "multiple" or "multiple copies" mentioned in the embodiments of this application all refer to a quantity of "at least two," for example, "multiple" means "at least two," and "multiple copies" means "at least two copies."

[0027] In one embodiment, such as Figure 2 As shown, a smart detection method based on headphones is provided, which can be applied to... Figure 1 Taking the headphones as an example, the explanation includes the following steps:

[0028] Step 202: Obtain the first reflected signal of the first light signal; wherein, the first light signal is the light signal emitted by the red light source on the dual-color light source substrate disposed in the earphone.

[0029] Specifically, the earphone contains a dual-color light source substrate and a red light photosensitive sensor. The dual-color light source substrate integrates a red light source for emitting red light. When earphone wearing detection is required, the red light source can be driven to emit a red light pulse. After the red light pulse illuminates the surrounding area, it will be reflected. At this time, the reflected red light can be received by the red light photosensitive sensor. For ease of description, the red light pulse emitted by the red light source during wearing detection will be referred to as the first light signal, and the reflected light received by the red light photosensitive sensor will be referred to as the first reflection signal.

[0030] In one embodiment, the red light source may specifically be a red LED (Light Emitting Diode).

[0031] In one embodiment, the earphones may specifically be TWS (True Wireless Stereo) earphones.

[0032] In one embodiment, hardware self-test and optical parameter calibration can be performed in the initial stage. The hardware self-test process is as follows: After power-on, the control module in the headset starts a self-test program to confirm the communication status of the dual-color light source substrate, red light photosensitive sensor, green light photosensitive sensor, and six-axis inertial sensor. The self-test program also detects the baseline noise value of the ambient light sensor. If the ambient light intensity is greater than a preset light intensity threshold, an anti-strong light mode is activated. In anti-strong light mode, if the light sensor (such as a photodiode) receives a very strong light signal, the gain of the ADC (Analog-to-Digital Converter) can be dynamically reduced to reduce the amplitude of the analog signal input to the ADC to a suitable range, avoiding saturation and ensuring accurate signal conversion. The optical parameter calibration process is as follows: (Refer to...) Figure 3 The red light source emits three pulses with a small current of 5mA and collects the reflected signal R_IR0 when the device is not worn, which serves as the environmental reflection reference. The green light source emits a single pulse with a current of 10mA and uses the reflected signal collected by the green light photosensitive sensor at this time as the original noise value V_noise for subsequent signal differential calculation. Figure 3 A schematic diagram of the pulse transmission time interval in one embodiment is shown.

[0033] Step 204: Perform headphone wearing detection based on the first reflected signal to obtain headphone wearing detection results.

[0034] Specifically, after obtaining the first reflected signal, headphone wearing detection can be performed based on the first reflected signal to determine whether the headphone is being worn. For example, since the first light signal is reflected when it encounters an object, part of the reflected light will return to the red light photosensitive sensor. If the headphone is worn on the ear, the first light signal will be reflected back by the skin and other tissues. However, if the headphone is not worn, the first light signal will be emitted towards the surrounding environment and reflected back by the surrounding environment. The amount of light reflected back by the skin and other tissues will be significantly different from the brightness of the light reflected back by the surrounding environment. That is, the first reflected signal collected by the red light photosensitive sensor when the headphone is worn will be significantly different from the first reflected signal collected by the red light photosensitive sensor when the headphone is not worn. Therefore, the first reflected signal is processed and analyzed, and the intensity and change of the first reflected signal are judged by a preset algorithm. If the intensity of the received first reflected signal reaches a certain threshold, it means that the headphone is being worn on the ear and is in a wearing state; if the intensity of the first reflected signal is lower than a certain threshold, for example, if the first reflected signal is close to R_IR0, it is determined that the headphone is not being worn.

[0035] Step 206: If it is determined that the headphones are in a wearing state based on the headphone wearing detection result, the second reflection signal of the second light signal is acquired; wherein, the second light signal is the light signal emitted by the green light source on the dual-color light source substrate.

[0036] Specifically, the earphones incorporate a dual-color light source substrate and a green light photosensitive sensor. The dual-color light source substrate integrates a green light source for emitting green light. The earphones drive the green light source to emit green light pulses, and the green light photosensitive sensor receives the reflected signals of these pulses. For ease of description, the green light pulses emitted by the green light source during heart rate detection will be referred to as the second light signal, and the reflected green light received by the green light photosensitive sensor will be referred to as the second reflected signal. When the earphones are determined to be worn based on the earphone wearing detection results, the green light photosensitive sensor receives the second reflected signal of the second light signal for subsequent heart rate detection.

[0037] In one embodiment, the green light source may specifically be a green LED (Light Emitting Diode).

[0038] In one embodiment, the red light source is initially driven at 10mA to detect ear canal fit. The fit coefficient is calculated using the reflected signal intensity R_IR, and the driving current of the green light source is dynamically adjusted. For example, the driving current of the green light source can be calculated using the formula: I_Green = 30mA × R_IR + 5mA. This driving current is used to avoid overexposure or underexposure of the second light signal.

[0039] Step 208: Perform heart rate detection based on the second reflected signal to obtain the heart rate detection result.

[0040] Specifically, when the second reflected signal is acquired, the headphones can perform heart rate detection based on it. For example, when the headphones perform heart rate detection, a green light source emits a second light signal that shines onto the skin surface of the ear. Part of the second light signal is absorbed by the skin, tissue, and blood, while the other part is reflected back. Because the heartbeat causes periodic changes in the blood volume in the arteries, when the heart contracts, the blood volume in the arteries increases, leading to increased absorption of the second light signal and decreased reflection; when the heart relaxes, the blood volume decreases, leading to decreased absorption of the second light signal and increased reflection. The green light photosensitive sensor converts the received changes in the second reflected signal into an electrical signal. Since the intensity of the second reflected signal changes periodically with the heartbeat, the converted electrical signal also exhibits corresponding periodic fluctuations. Therefore, the heart rate can be determined based on the periodic fluctuations in the electrical signal, for example, by using the time interval between adjacent peaks or troughs as the pulse cycle, based on the formula: heart rate = 60 / pulse cycle.

[0041] In the aforementioned intelligent detection method based on headphones, headphone wearing detection can be performed by acquiring the first reflected signal of the first light signal; when the headphone wearing detection result determines that the headphone is being worn, heart rate detection can be performed by acquiring the second reflected signal of the second light signal. Because it can perform both headphone wearing detection and heart rate detection, compared to traditional methods that only play audio, it greatly enriches the headphone's functionality and improves its utilization. Furthermore, since the red and green light sources are integrated on a dual-color light source substrate, compared to deploying separate red and green light sources, the additional space occupied by these sources inside the headphone can be reduced, aligning with the trend of headphone miniaturization design.

[0042] In one embodiment, the red light source and the green light source are integrated and packaged on a dual-color light source substrate. The red light source and the green light source are driven by a time-division driving circuit so that the red light source and the green light source alternately emit light signals. The first light signal emitted by the red light source and the second light signal emitted by the green light source are focused by a microlens array to form a light spot, and the light spot covers the contact area between the concha cavity and the ear canal. The first reflected signal is collected by a red light photosensitive sensor set in the earphone, and the second reflected signal is collected by a green light photosensitive sensor set in the earphone. The red light photosensitive sensor, the green light photosensitive sensor and the dual-color light source substrate form a triangular layout.

[0043] Specifically, a red light source (wavelength 850±10nm) and a green light source (wavelength 550±5nm) are integrated onto a substrate of a predetermined size using a COB (Chip-on-Board) process. Furthermore, a microlens array is incorporated within the earphone. The first light signal emitted by the red light source and the second light signal emitted by the green light source are focused at a 30° divergence angle through the microlens array to form a light spot that covers the area where the concha and ear canal meet.

[0044] The dual-color light source substrate in the headphones is driven by a time-division driving circuit, which alternately activates the red and green light sources on the dual-color light source substrate at a preset frequency. (The reference value is missing from the original text.) Figure 4 The first light signal emitted by the red light source has a pulse width of 2ms (50% duty cycle) and is used for wear detection. The second light signal emitted by the green light source has a pulse width of 4ms (100% duty cycle) and is used for heart rate detection. The two channels are separated by a 0.5ms blanking period to avoid crosstalk. Figure 4 A schematic diagram of a time-division drive waveform is shown in one embodiment.

[0045] refer to Figure 5 Two sets of photosensitive sensors are symmetrically arranged in the earphone. These two sets of photosensitive sensors include a red light photosensitive sensor and a green light photosensitive sensor. The red light photosensitive sensor is covered with a narrow-band infrared filter (850nm±5nm, transmittance>90%), and the green light photosensitive sensor is covered with a green light filter (550nm±3nm, cutoff depth-40dB). There is a certain distance between the red light photosensitive sensor and the green light photosensitive sensor, for example, the distance between the red light photosensitive sensor and the green light photosensitive sensor can be 1.8mm. The center of the red light photosensitive sensor, the center of the green light photosensitive sensor and the center of the dual-color light source substrate form a golden triangle layout to optimize the reflected light reception efficiency and improve the detection accuracy of wear detection and heart rate detection. Figure 5 A schematic diagram of the device layout in one embodiment is shown.

[0046] In this embodiment, the red light source and the green light source are integrated on a dual-color light source substrate (the red light source is used for wear detection, and the green light source is used for heart rate detection), which saves extra space for the red light driving circuit and the filter, allowing the headphones to be miniaturized.

[0047] Compared to using a single red light for both wear detection and heart rate monitoring, infrared light, while capable of penetrating the skin to detect wear status, has low sensitivity to capillary blood flow signals and is easily affected by slow changes in venous blood volume, resulting in lower accuracy in heart rate detection. In contrast, this embodiment uses green light for heart rate detection. Green light is more sensitive to changes in capillary blood flow and has a wider dynamic range in motion scenarios. It is located in the "spectral valley" region where ambient light intensity is low. Combined with a narrowband filter, it can reduce ambient noise by more than 60%. Therefore, using green light for heart rate detection can improve the accuracy of heart rate detection.

[0048] Furthermore, while green light with a wavelength of 530nm can be used for heart rate detection, its signal-to-noise ratio from reflected light on the uneven surface of the ear canal is more than 60% lower than that measured by wrist, resulting in lower accuracy. This embodiment of the application adjusts the wavelength of the green light (i.e., the second optical signal) from 530nm to 550nm, which not only improves the signal-to-noise ratio of the reflected signal to enhance the accuracy of heart rate detection, but also allows the effective detection depth of green light at this wavelength in ear canal tissue to reach 1.5mm, thus meeting the detection requirements of deep blood vessels and enabling heart rate detection even when facing deep blood vessels.

[0049] In one embodiment, headphone wearing detection based on a first reflected signal includes: acquiring acceleration data collected by an accelerometer installed in the headphone; determining the average value of the first reflected signal based on the first reflected signal of the first light signal emitted by the red light source in multiple cycles; determining a current wearing detection threshold based on the acceleration data and the average value of the first reflected signal; and performing headphone wearing detection based on the first reflected signal of the first light signal emitted in multiple cycles and the current wearing detection threshold.

[0050] Specifically, the headphones can also be equipped with an accelerometer to collect acceleration data. The headphones can acquire the Z-axis data from the accelerometer and determine the change in ear canal pressure based on this data. For example, the headphones can use the Z-axis data from the accelerometer as the change in ear canal pressure, or the headphones can input the Z-axis data from the accelerometer into an ear canal pressure change calculation model, which then outputs the change in ear canal pressure. Furthermore, since the red and green light sources, driven by a time-division multiplexing circuit, alternately emit light signals, the process of the red light source emitting a first light signal and the green light source emitting a second light signal is considered as a cycle. The first reflected signals of the first light signals emitted by the red light source within multiple consecutive cycles can be acquired, and the average of the acquired first reflected signals can be calculated to obtain the average value of the first reflected signals.

[0051] Furthermore, the earphone inputs the change in ear canal pressure and the average value of the first reflected signal into the wear detection threshold calculation model, which then outputs the current wear detection threshold. Based on the first reflected signal of the first light signal emitted over multiple cycles and this current wear detection threshold, the earphone can perform wear detection to determine whether it is being worn.

[0052] In one embodiment, the wear detection threshold calculation model can be specifically defined by the following formula: Thres_IR(t) = α × R_IR_avg + β × (ΔP / 0.1g). Where Thres_IR(t) is the current wear detection threshold, R_IR_avg is the average value of the first reflected signal, ΔP is the change in ear canal pressure, and α and β are preset parameters that can be freely adjusted according to requirements. For example, α can be 0.6 and β can be 0.4, so that every 0.1g change in ΔP corresponds to an adjustment of ±5% in the wear detection threshold.

[0053] In the above embodiments, since different people have different ear sizes, and the difference in ear size will lead to changes in ear pressure, it is necessary to set a dynamic wearing detection threshold so that the wearing status detected based on the dynamic wearing detection threshold can be more accurate.

[0054] In one embodiment, headphone wearing detection is performed based on the first reflection signal of the first light signal emitted in multiple cycles and a wearing detection threshold, including: when the first reflection signal of the first light signal emitted in multiple cycles is greater than the wearing detection threshold, determining the standard deviation of the first reflection signal; when the standard deviation is less than a preset standard deviation threshold, determining that the headphone is in a wearing state.

[0055] Specifically, when the first reflected signal of the first light signal emitted within multiple cycles is greater than the wear detection threshold—for example, when the first reflected signal of the first light signal emitted within five consecutive cycles is greater than the wear detection threshold—the headphones are considered to be possibly in a wearing state. In this case, the standard deviation of the first reflected signal of the first light signal emitted within multiple cycles can be determined. When this standard deviation is less than a preset standard deviation threshold, the headphones are determined to be in a wearing state; for example, when the standard deviation is less than 3%, the headphones are determined to be in a wearing state.

[0056] In one embodiment, if it is detected that the headphone is being worn but the signal strength of the first reflected signal is lower than a preset signal threshold, an abnormal wearing alarm is triggered. For example, a mobile application may be triggered to display a reminder that the headphones are being worn incorrectly, so as to prompt the user to wear the headphones correctly.

[0057] In one embodiment, a wearing abnormality alarm is triggered when R_IR < 0.7 × R_IR0, where R_IR is the first reflected signal, R_IR0 is the reflected signal in the unworn state determined during environmental parameter calibration, and 0.7 × R_IR0 is a preset signal threshold. The wearing abnormality alarm can be triggered when the intensity of the first reflected signal is less than the preset signal threshold for one or more consecutive periods. By issuing wearing abnormality alarms, users can be aware that the headphones are in an abnormal wearing state and adjust their headphones accordingly, thus greatly improving the user experience.

[0058] In the above embodiments, by determining the standard deviation of multiple first reflection signals when the first reflection signals of the first light signals emitted in multiple cycles are all greater than the current wearing detection threshold, the signal fluctuation rate can be determined based on the standard deviation. Thus, the earphone is determined to be in a wearing state only when multiple first reflection signals are all greater than the current wearing detection threshold and the signal fluctuation rate is small, which can improve the accuracy of earphone wearing detection.

[0059] In one embodiment, heart rate detection based on the second reflected signal to obtain a heart rate detection result includes: converting the second reflected signal into an electrical signal; amplifying and filtering the electrical signal to obtain a first photoplethysmography (PPG) signal; determining an initial heart rate based on the peaks and valleys in the first PPG signal; and correcting the initial heart rate based on the first reflected signal to obtain a target heart rate.

[0060] Specifically, the headphones can convert the second reflected signal into an electrical signal and amplify it. This electrical signal is specifically a voltage signal. Further, the headphones can use a complex Morlet wavelet (a wavelet with a center frequency of 1Hz and a bandwidth of 0.8Hz) to perform time-frequency analysis on the amplified electrical signal, extracting the dominant frequency to obtain a photoplethysmography (PPG) signal. For ease of description, the PPG signal obtained during heart rate detection will be referred to as the first PPG signal. The headphones dynamically track the first derivative (dPPG / dt) of the first PPG signal to identify peaks and troughs, determining the initial heart rate based on these peaks and troughs. Further, the headphones correct the initial heart rate based on the first reflected signal of the first light signal emitted by the red light source to obtain a target heart rate, which is the final obtained heart rate.

[0061] In one embodiment, the programmable gain amplifier in the headphones can dynamically adjust the gain to amplify the electrical signal, ensuring that the ADC (Analog-to-Digital Converter) input range is within a preset range, such as 0.8-2.4V.

[0062] In one embodiment, the correction of the initial heart rate based on the first reflected signal of the first light signal emitted by the red light source by the headphones includes: when the difference between the first reflected signals of multiple consecutive cycles is less than a preset difference threshold, the first reflected signal is considered relatively stable. When the first reflected signal is relatively stable, the current environment can be considered relatively stable, and the initial heart rate can be directly used as the target heart rate without correction; when the difference between the first reflected signals of multiple consecutive cycles is less than the preset difference threshold, the first reflected signal is considered unstable. When the first reflected signal is unstable, the current environment can be considered relatively complex with many influencing factors (e.g., the influence of ambient light). In this case, the influence of influencing factors on the heart rate can be removed. For example, ambient light can be removed, and the time used to determine the heart rate can be lengthened. For example, the target heart rate can be calculated based on the second reflected signal collected in two cycles, instead of the original one cycle.

[0063] In one embodiment, the earphone also includes a six-axis inertial sensor to acquire motion data collected by the sensor. Based on this motion data, a personalized motion noise spectrum template is constructed. For example, different motion data correspond to different operating states, and different motion states correspond to different personalized motion noise spectrum templates. Once the personalized motion noise spectrum template is obtained, the electrical signal can be filtered based on it. Because the electrical signal can be filtered using the personalized motion noise spectrum template, different filtering schemes can be executed under different motion states, resulting in more accurate filtering and, consequently, a more accurate target heart rate.

[0064] In the above embodiments, by amplifying and filtering the electrical signal, a clearer and more accurate first photoplethysmography (PPG) signal can be obtained. This clearer and more accurate PPG signal can improve the accuracy of the initial heart rate, thereby improving the accuracy of the target heart rate. Furthermore, by correcting the initial heart rate, the accuracy of the final target heart rate can be further improved.

[0065] In one embodiment, the method further includes: for each sampling current within a preset current range, triggering a green light source to emit a third light signal under the drive of the sampling current; determining a second photoplethysmography (PPG) signal corresponding to the sampling current based on the third reflection signal of the third light signal emitted under the sampling current; determining a target current value and ear blood vessel depth based on the second PPG signal corresponding to each sampling current; and determining a driving mode of the green light source based on the target current value and ear blood vessel depth.

[0066] Specifically, optical path optimization can be performed before heart rate detection. During optical path optimization, a preset current range can be determined, for example, from 5 mA to 30 mA. The current within this preset range is sampled to obtain a sampled current. For example, for the 5 mA to 30 mA range, 5 mA can be used as the starting value, increasing by 1 mA each time until 30 mA is reached. Further, for each sampled current, a green light source can be triggered to emit green light under the drive of the sampled current. For ease of description, the green light emitted by the green light source during optical path optimization will be referred to as the third optical signal. The reflected signal of the third optical signal can be acquired, and the photoplethysmography (PPG) signal corresponding to the sampled current can be determined based on this reflected signal. For ease of description, the reflected signal of the third signal will be referred to as the third reflected signal, and the PPG signal determined based on the third reflected signal will be referred to as the second PPG signal.

[0067] Once the second photoplethysmography (PPG) signal corresponding to each sampling current is determined, the signal-to-noise ratio (SNR) of each PPG signal can be determined, and the target current value and the depth of the ear's blood vessels can be determined based on the SNR. For example, since a high SNR means a clearer signal and less noise interference, the second PPG signal with the highest SNR can be determined, and the sampling current corresponding to this second PPG signal can be used as the target current. Subsequently, the green light source can be driven to emit a second light signal for heart rate acquisition based on this target current. Furthermore, superficial blood vessels are easily affected by external factors, such as skin movement, changes in ambient light, and changes in electrode-skin contact. These interferences introduce more noise, resulting in a lower SNR for the PPG signal obtained from superficial blood vessels. In contrast, the tissue surrounding deep blood vessels is relatively stable, and external interference factors have less impact on them. Therefore, the PPG signal obtained from deep blood vessels contains relatively less noise, resulting in a higher SNR. Therefore, for the second photoplethysmography (PPG) signal with the highest signal-to-noise ratio (SNR), it can be determined whether the SNR of the second PPG signal is greater than a preset SNR threshold. If it is greater, the ear blood vessels are determined to be deep vessels; if it is less, the ear blood vessels are determined to be superficial vessels. For deep vessels, short-pulse-width green light is used to improve penetration, that is, the pulse width of the second light signal is shortened to less than the first preset pulse width threshold. For superficial vessels, long-pulse-width green light is used to improve the accuracy of heart rate measurement, that is, the pulse width of the second light signal is increased to greater than the second preset pulse width threshold. The first preset pulse width threshold is less than the second preset pulse width threshold.

[0068] In the above embodiments, the accuracy of heart rate detection can be improved by adjusting the driving mode of the green light source.

[0069] In one embodiment, reference Figure 6 , Figure 6A schematic diagram of the overall framework flow in one embodiment is shown. First, system initialization is performed, followed by environmental parameter calibration. This calibration process determines the reflected signal R_IR0 and the original noise value V_noise in the unworn state. The original noise value can be differentially calculated during heart rate detection to obtain a noise-removed signal. After environmental parameter calibration, a wear detection threshold is determined based on the first light signal emitted by the red light source. Wear detection is then performed based on the determined wear detection threshold and the first light signal. When it is determined that the user is wearing the headphones, a fit detection is performed to adjust the driving current value of the green light source. After adjusting the driving current value of the green light source, heart rate detection is initiated. Following the initiation of heart rate detection, optical path optimization is performed to optimize the driving mode of the green light source, enabling it to emit a second light signal under the optimized driving mode. The headphones can establish a personalized motion noise spectrum template based on motion data collected by a six-axis inertial sensor. Heart rate detection is performed using this personalized motion noise spectrum template and the second reflected signal of the second light signal. During wear detection and continuous heart rate detection, the earphone drives the red light source and the green light source in a time-division driving manner, so that the first light signal and the second light signal can be emitted alternately, and the first light signal and the second light signal are processed in real time.

[0070] In one embodiment, reference Figure 7 , Figure 7 A schematic diagram of the workflow in one embodiment is shown. Phase 1: System Initialization and Environmental Calibration. System initialization and environmental calibration include: 1. Hardware Self-Test: After power-on, the control module starts a self-test program to confirm the communication status of the dual-color light source substrate, the photosensitive sensor array, and the six-axis inertial sensor. It detects the baseline noise value of the ambient light sensor; if the ambient light intensity is greater than a preset light intensity threshold, it activates the anti-strong light mode. 2. Optical Parameter Calibration: The red light source emits a preset number of pulses with a preset small current, collecting the reflected signal R_IR0 in the unworn state as the environmental reflection reference. The green light source emits a single pulse with a preset current intensity, recording the original noise value V_noise of the photosensitive sensor for subsequent signal differential calculation.

[0071] Phase Two: Wearing Detection: A red light source emits pulses at a preset frequency (e.g., driving current 10mA, pulse width 2ms), simultaneously acquiring the first reflected signal R_IR. Combined with accelerometer Z-axis data (ear pressure change ΔP), the current wearing detection threshold is calculated. If the first reflected signal R_IR exceeds the current wearing detection threshold Thres_IR for multiple consecutive cycles and the signal volatility exceeds a preset standard deviation threshold, it is considered valid wearing. If wearing is detected but the first reflected signal strength is below a preset signal strength threshold, such as below 0.7R_IR0, a wearing abnormality alarm is triggered.

[0072] Phase 3: Heart Rate Monitoring Initiation: 1. Optical Path Optimization: The green light source emits test pulses with a progressive current (e.g., 5mA→30mA), and the signal-to-noise ratio (SNR) of the second photoplethysmography (PPG) wave signal is calculated in real time. The optimal light source driving mode is selected based on the SNR of the second PPG wave signal. 2. Motion Baseline Establishment: The six-axis inertial sensor is simultaneously activated to collect pre-motion data to construct a personalized motion noise spectrum template for subsequent adaptive filtering.

[0073] Phase Four: Continuous Heart Rate Monitoring: 1. Time-Division Driving and Signal Acquisition: Timing control defines a complete cycle as each preset duration, for example, every 4ms. There is a preset time interval between the red and green light source emissions, for example, at least 1ms. Simultaneously, it ensures strict alignment between the dual-color light source substrate driving and ADC sampling, with timing errors less than a preset timing error threshold, for example, less than 10μs. 2. A transimpedance amplifier (e.g., 1kHz bandwidth) converts the second optical signal into an electrical signal. A programmable gain amplifier (PGA) dynamically adjusts the gain to ensure the ADC input range is within a preset range, for example, 0.8-2.4V. The algorithm layer uses wavelet functions (e.g., wavelets with a center frequency of 1Hz and a bandwidth of 0.8Hz) for time-frequency analysis to extract the dominant frequency and obtain the first photoplethysmography (PPG) signal. The first derivative (dPPG / dt) of the first PPG signal is dynamically tracked to identify peaks and troughs, and the initial heart rate is determined based on these peaks and troughs. After obtaining the initial heart rate, it can be further corrected by combining it with the first optical signal to obtain the target heart rate.

[0074] Phase 5: Low-Power Mode Switching: If the heart rate fluctuation rate remains below a preset threshold (e.g., less than 5%) for a continuously preset duration, such as 5 minutes, the user is determined to be in a resting state. At this time, the headphones automatically switch to low-power mode, reducing the green light sampling rate and the driving current of the dual-color light source substrate, while simultaneously disabling the six-axis inertial sensor function. Deep sleep mode is entered when the headphones are detected as being removed (the first reflected signal drops sharply to the R_IR0 level).

[0075] The beneficial effects of the above embodiments are as follows:

[0076] 1. Hardware Integration and Space Constraints: Integrating both red and green light sources saves additional space compared to red light driving circuitry and filters. Adjusting existing green light parameters requires no additional hardware; deep detection can be achieved directly through software / circuit optimization. Furthermore, the 550nm green light achieves an effective detection depth of up to 1.5mm in ear canal tissue, meeting the needs of deep blood vessels, and boasts a 40% higher signal-to-noise ratio (SNR) than red light, improving the accuracy of heart rate detection.

[0077] 2. Signal quality and anti-interference capability: Green PPG signal is more sensitive to changes in capillary blood flow (especially suitable for heart rate monitoring). In sports scenarios, the dynamic range of green light signal is larger and it is located in the "spectral valley" where the ambient light intensity is low. When combined with a narrowband filter, it can reduce ambient noise by more than 60%, further improving the accuracy of heart rate detection.

[0078] 3. System Power Consumption and Energy Efficiency Optimization: The electro-optical conversion efficiency of green light source is significantly higher than that of red light, and the driving current of green light can be reduced by 50% under the same light output power. This reduces heat generation and avoids temperature rise in the ear canal affecting comfort. The wavelength difference between infrared and green light reaches 300nm, resulting in higher spectral isolation and a crosstalk rate of <1% during time-division driving. If a combination of red and green light is used instead, the wavelength difference is only 120nm, requiring more complex filtering design or timing isolation, increasing power consumption and cost.

[0079] 4. Algorithm-level compensation enhancement: Since the vessel depth can be inferred, the driving mode of the green light can be adjusted in a timely manner based on the inferred vessel depth, thereby improving the accuracy of heart rate detection. Simultaneously, by combining infrared reflection signals and six-axis inertial sensor data, the heart rate detection results are dynamically corrected, further enhancing the accuracy of heart rate detection.

[0080] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0081] Based on the same inventive concept, this application also provides a headphone-based intelligent detection device for implementing the headphone-based intelligent detection method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more headphone-based intelligent detection device embodiments provided below can be found in the limitations of the headphone-based intelligent detection method described above, and will not be repeated here.

[0082] In one embodiment, such as Figure 8 As shown, a smart detection device 800 based on headphones is provided, including: a wear detection module 802 and a heart rate detection module 804, wherein:

[0083] The wearing detection module 802 is used to acquire the first reflected signal of the first light signal; wherein, the first light signal is the light signal emitted by the red light source on the dual-color light source substrate in the earphone; the earphone wearing detection is performed based on the first reflected signal to obtain the earphone wearing detection result;

[0084] The heart rate detection module 804 is used to acquire a second reflected signal of a second light signal when the headphones are determined to be in a wearing state based on the headphone wearing detection result; wherein the second light signal is the light signal emitted by the green light source on the dual-color light source substrate; and heart rate detection is performed based on the second reflected signal to obtain a heart rate detection result.

[0085] In one embodiment, the red light source and the green light source are integrated and packaged on a dual-color light source substrate. The red light source and the green light source are driven by a time-division driving circuit so that the red light source and the green light source alternately emit light signals. The first light signal emitted by the red light source and the second light signal emitted by the green light source are focused by a microlens array to form a light spot, and the light spot covers the contact area between the concha cavity and the ear canal. The first reflected signal is collected by a red light photosensitive sensor set in the earphone, and the second reflected signal is collected by a green light photosensitive sensor set in the earphone. The red light photosensitive sensor, the green light photosensitive sensor and the dual-color light source substrate form a triangular layout.

[0086] In one embodiment, the wear detection module 802 is further configured to acquire acceleration data collected by an accelerometer installed in the earphone; determine the average value of the first reflection signal based on the first reflection signal of the first light signal emitted by the red light source in multiple cycles; determine the current wear detection threshold based on the acceleration data and the average value of the first reflection signal; and perform earphone wear detection based on the first reflection signal of the first light signal emitted in multiple cycles and the current wear detection threshold.

[0087] In one embodiment, the wear detection module 802 is further configured to determine the standard deviation of the first reflected signal when the first reflected signal of the first light signal emitted in multiple cycles is greater than the current wear detection threshold; and to determine that the earphone is in a wearing state when the standard deviation is less than a preset standard deviation threshold.

[0088] In one embodiment, the heart rate detection module 804 is further configured to convert the second reflected signal into an electrical signal; amplify and filter the electrical signal to obtain a first photoplethysmography (PPG) signal; determine the initial heart rate based on the peaks and valleys in the first PPG signal; and correct the initial heart rate based on the first reflected signal to obtain a target heart rate.

[0089] In one embodiment, the heart rate detection module 804 is further configured to, for each sampled current within a preset current range, trigger the green light source to emit a third light signal under the drive of the sampled current; determine the second photoplethysmography (PPG) signal corresponding to the sampled current based on the third reflection signal of the third light signal emitted under the sampled current; determine the target current value and the depth of the ear blood vessels based on the second PPG signal corresponding to each sampled current; and determine the driving mode of the green light source based on the target current value and the depth of the ear blood vessels.

[0090] The modules in the aforementioned headphone-based intelligent detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0091] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0092] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0093] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0094] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0095] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, etc., and are not limited to these.

[0096] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0097] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A smart detection method based on headphones, characterized in that, The method includes: Acquire a first reflected signal of the first optical signal; wherein, the first optical signal is an optical signal emitted by a red light source disposed on a dual-color light source substrate in the earphone; Based on the first reflected signal, headphone wearing detection is performed to obtain headphone wearing detection results; When it is determined that the headphones are in a wearing state based on the headphone wearing detection result, a second reflected signal of the second light signal is acquired; wherein, the second light signal is the light signal emitted by the green light source on the dual-color light source substrate; Heart rate detection is performed based on the second reflected signal to obtain the heart rate detection result.

2. The method according to claim 1, characterized in that, The red light source and the green light source are integrated and packaged on a dual-color light source substrate. The red light source and the green light source are driven by a time-division driving circuit so that the red light source and the green light source alternately emit light signals. The first light signal emitted by the red light source and the second light signal emitted by the green light source are focused at a divergence angle through a microlens array to form a light spot, and the light spot covers the contact area between the concha cavity and the ear canal. The first reflected signal is acquired by a red light photosensitive sensor installed in the earphone, and the second reflected signal is acquired by a green light photosensitive sensor installed in the earphone. The red light photosensitive sensor, the green light photosensitive sensor and the dual-color light source substrate form a triangular layout.

3. The method according to claim 1, characterized in that, The headphone wearing detection based on the first reflected signal includes: Acquire acceleration data collected by the accelerometer sensor installed in the earphone; The average value of the first reflected signal is determined based on the first reflected signal of the first light signal emitted by the red light source in multiple cycles; Based on the acceleration data and the average value of the first reflected signal, the current wearing detection threshold is determined; Earphone wearing detection is performed based on the first reflected signal of the first light signal emitted within the multiple cycles and the current wearing detection threshold.

4. The method according to claim 3, characterized in that, The headphone wearing detection is performed based on the first reflected signal of the first light signal emitted within the multiple cycles and the current wearing detection threshold, including: When the first reflected signal of the first light signal emitted in the plurality of cycles is greater than the current wearing detection threshold, the standard deviation of the first reflected signal is determined. When the standard deviation is less than a preset standard deviation threshold, the headphones are determined to be in a wearing state.

5. The method according to claim 1, characterized in that, The step of detecting heart rate based on the second reflected signal to obtain heart rate detection results includes: The second reflected signal is converted into an electrical signal; The electrical signal is amplified and filtered to obtain a first photoplethysmography (PPG) signal. The initial heart rate is determined based on the peaks and valleys in the first photoplethysmography signal. The initial heart rate is corrected based on the first reflected signal to obtain the target heart rate.

6. The method according to claim 1, characterized in that, The method further includes: For each sampled current within a preset current range, the green light source is triggered to emit a third light signal under the drive of the sampled current; Based on the third reflected signal of the third optical signal emitted under the sampling current, the second photoplethysmography signal corresponding to the sampling current is determined. Based on the second photoplethysmography signal corresponding to each of the sampling currents, the target current value and the depth of the ear blood vessels are determined. The driving mode of the green light source is determined based on the target current value and the depth of the ear blood vessels.

7. A smart detection device based on headphones, characterized in that, The device includes: The wearing detection module is used to acquire a first reflected signal of a first light signal; wherein, the first light signal is a light signal emitted by a red light source on a dual-color light source substrate disposed in the earphone; the earphone wearing detection is performed based on the first reflected signal to obtain an earphone wearing detection result; The heart rate detection module is used to acquire a second reflected signal of a second light signal when the earphone is determined to be in a wearing state based on the earphone wearing detection result; wherein the second light signal is the light signal emitted by the green light source on the dual-color light source substrate; and to perform heart rate detection based on the second reflected signal to obtain a heart rate detection result.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images