Earphone state detection method and device based on multiple sensors

Abstract

The embodiment of the invention discloses an earphone state detection method and device based on multiple sensors. The earphone has a horn located within the ear canal, a first pickup sensor located within the ear canal and disposed adjacent to the horn, and a second pickup sensor located outside the ear canal. The method comprises the following steps: acquiring first earphone state information according to a source audio signal input to the loudspeaker and a first audio signal picked up by the first pickup sensor; acquiring second earphone state information according to a second audio signal picked up by the second pickup sensor and a first audio signal picked up by the first pickup sensor; and outputting a final detection result of the earphone state based on the first earphone state information and the second earphone state information. According to the method, the feature relationship between the two types of signals is combined, the earphone state can be effectively divided in a complex environment, and the accuracy of earphone state detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wearing status detection of wearable devices, and in particular to a method and device for detecting the status of an earphone based on multiple sensors. Background Art

[0002] Headphones are increasingly used in our daily lives because of their small size and portability. For example, we use headphones to listen to music and watch movies, etc. It can be seen that the listening effect of headphones is very important to users. Most manufacturers pay more attention to the quality of the headphones themselves, but ignore the wearing state of the headphones, that is, the coupling state of the headphones and the ear canal, and the impact on the listening effect. If the headphones are worn loosely, the coupling between the headphones and the ear canal is not good, and the low frequency will leak, seriously affecting the listening effect of the low frequency. If the headphones are worn tightly, the coupling between the headphones and the ear canal is good, the low frequency is maintained, and users can feel a better listening effect.

[0003] In addition, for active noise reduction headphones, the coupling between the headphones and the ear canal will also affect the noise reduction effect, so it is also hoped to select appropriate noise reduction filters under different coupling conditions to obtain better noise reduction effects, or perform audio compensation, etc. In particular, when the headphones are in an abnormal state, such as being placed in a pocket or held in the hand, the headphones are squeezed, which may cause the headphones to howl, which is also a kind of noise pollution to the human ear, so when the headphones are in an abnormal state, it is also hoped to avoid the generation of howling.

[0004] Existing earphone wearing state detection methods, for example, can embed infrasound signals that are not easily detected by the human ear in the audio signal sent to the speaker / loudspeaker, and then use the microphone in the ear canal to detect the amplitude of the infrasound, so as to determine the current low-frequency leakage situation, thereby detecting the wearing state. There is also a method of using the source audio signal and the audio signal collected by the feedback microphone to detect the wearing state according to the difference of the weighted sum of their low-frequency amplitudes. These methods are nothing more than using the change of the absolute amplitude of the low-frequency band to determine the wearing state, and their defects are poor noise resistance (when the external noise is large, it will bring great difficulties to the detection), and poor performance in adapting to different people and different scenes. In addition, these methods basically use a single relationship between two signals (the input audio signal and the signal collected by the microphone in the ear canal) to detect the state, but in actual applications, the earphone is in a complex scene, for example, the audio signal played may be of various kinds, and the external noise conditions may also be of various kinds; there may be a situation where there is no audio signal, or it may be in a very noisy environment, and at this time, only using a single relationship between the two signals cannot obtain an effective wearing state. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for detecting the state of an earphone based on multiple sensors, which aims to improve the accuracy of detecting the state of an earphone in various complex scenarios.

[0006] According to a first aspect of the present invention, a method for detecting a state of an earphone based on multiple sensors is provided, wherein the earphone comprises a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal, the method comprising:

[0007] Acquire first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by the first sound pickup sensor;

[0008] Acquire second earphone state information according to the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor;

[0009] Based on the first earphone state information and the second earphone state information, a final detection result of the earphone state is output.

[0010] According to a second aspect of the present invention, a multi-sensor based earphone state detection device is provided, wherein the earphone has a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal, and the device comprises:

[0011] A first state acquisition module, configured to acquire first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by the first sound pickup sensor;

[0012] A second state acquisition module, used for acquiring second earphone state information according to a second audio signal picked up by the second sound pickup sensor and a first audio signal picked up by the first sound pickup sensor;

[0013] The state fusion output module is used to output a final detection result of the headphone state based on the first headphone state information and the second headphone state information.

[0014] According to a third aspect of the present invention, an earphone is provided, comprising a memory and a processor, and further comprising a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal; a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the aforementioned multi-sensor based earphone status detection method.

[0015] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more computer programs, and when the one or more computer programs are executed by a processor, the aforementioned multi-sensor based earphone status detection method is implemented.

[0016] The beneficial effects of the embodiments of the present invention are:

[0017] Each embodiment of the present invention combines the relationship between two types of signals to detect the status of the headphone, not only utilizing the characteristic relationship between the signal input to the speaker and the signal picked up by the first sound pickup sensor, which can characterize the leakage of the headphone in low-frequency conditions, and obtain the first headphone status information, but also utilizing the characteristic relationship between the signal picked up by the second sound pickup sensor and the signal picked up by the first sound pickup sensor, which can characterize the sound insulation effect of the headphone in medium and high frequency conditions, and obtain the second headphone status information, and then fuse the first headphone status information and the second headphone status information to output the final detection result of the headphone status. Since the characteristic relationship between the two types of signals is combined when performing the headphone status detection, the headphone status can be effectively divided in a complex environment, thereby improving the accuracy of the headphone status detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings. In the drawings:

[0019] Figure 1 A schematic diagram showing the structure of earphones in various embodiments of the present invention is shown;

[0020] Figure 2 A schematic diagram showing a flow chart of a method for detecting a headset state based on multiple sensors according to an embodiment of the present invention;

[0021] Figure 3 The amplitude-frequency response of the transfer function between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor in different states is shown;

[0022] Figure 4 The phase-frequency response of the transfer function between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor in different states is shown;

[0023] Figure 5 The amplitude-frequency response of the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor in different states is shown;

[0024] Figure 6 The phase-frequency response of the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor in different states is shown;

[0025] Figure 7 A schematic diagram of a process for obtaining first headset status information according to an embodiment of the present invention is shown;

[0026] Figure 8 Shows the corresponding Figure 7 Schematic diagram of the logical judgment process;

[0027] Fig. 9 A schematic diagram showing a process of obtaining second headset status information according to an embodiment of the present invention;

[0028] Fig.10 Shows the corresponding Fig. 9 Schematic diagram of the logical judgment process;

[0029] Fig.11 A schematic diagram of a process of fusing and outputting two types of earphone status information according to an embodiment of the present invention is shown;

[0030] Fig.12 Shows the corresponding Fig.11 Schematic diagram of the logical judgment process;

[0031] Fig.13 A schematic structural diagram of a headset state detection device based on multiple sensors according to an embodiment of the present invention is shown;

[0032] Fig.14 A schematic diagram showing the structure of a first state acquisition module according to an embodiment of the present invention is shown;

[0033] Fig.15 A schematic diagram showing the structure of a second state acquisition module according to an embodiment of the present invention is shown;

[0034] Fig.16 A schematic diagram showing the structure of a state fusion output module according to an embodiment of the present invention is shown;

[0035] Fig.17 A schematic structural diagram of an earphone according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0036] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. These embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein.

[0037] Figure 1 Schematic diagram of the structure of the earphone in each embodiment of the present invention is shown. Figure 1 The earphones in the following embodiments of the present invention include a speaker 10 located in the ear canal, a first sound pickup sensor 20 located in the ear canal and arranged near the speaker 10, and a second sound pickup sensor 30 located outside the ear canal, wherein the speaker 10 is an electroacoustic converter, and the speaker 10 located in the ear canal and the first sound pickup sensor 20 are connected to the outside through a sound-permeable hole 40 on the earphone shell.

[0038] Based on the above positional relationship, the first sound pickup sensor 20 is used to pick up the sound signal in the ear canal (including external noise leaking into the ear canal, and the sound signal played by the headphone speaker, etc.), and the second sound pickup sensor is used to pick up the external noise signal. In addition, according to the signal relationship between the source audio signal input to the speaker 10 and the audio signal picked up by the first sound pickup sensor 20, the leakage of the earphone in the low-frequency state can be characterized, and according to the signal relationship between the audio signal picked up by the second sound pickup sensor 30 and the audio signal picked up by the first sound pickup sensor 20, the sound insulation of the earphone in the medium and high-frequency state can be characterized.

[0039] Different couplings between earphones and ear canals will show different characteristics in the earphone system. For example, in normal wearing conditions, when the coupling is good, the cavity formed by the earphones and ear canals has good sealing, the earphones have basically no leakage at low frequencies, and the earphones have good sound insulation at mid- and high-frequency levels; when the coupling is not good, the cavity formed by the earphones and ear canals has poor sealing, the earphones have greater attenuation at low frequencies, and the degree of coupling is different, the attenuation is different, and the earphones have poor sound insulation at mid- and high-frequency levels. In some abnormal conditions, such as when the earphones are not worn, the earphones are placed on the table, and the sound outlet is completely open; for example, when the earphones are held in the hand, the earphones are in a very small cavity and may produce howling, etc., the characteristic relationship between the signals is also obviously different from normal wearing. For example, when it is fully open, the low-frequency amplitude will be lower, and when it howls, the high-frequency amplitude is very high, and the phase is obviously beyond the range of normal wearing.

[0040] If it is determined that the current headset is in an abnormal howling situation, various means can be used to suppress the unwanted howling, such as controlling the ANC (Active Noise Reduction) switch or controlling the gain of the noise reduction filter to avoid howling in an abnormal state, or turning off the ANC or reducing the noise reduction gain in an abnormal state.

[0041] In order to accurately detect the status of the headset in various complex scenarios, so as to perform a series of headset controls, such as ANC switch, ANC filter adjustment and audio compensation, etc. The present invention combines multiple sensors to detect the status of the headset.

[0042] Figure 2 FIG. 1 is a flow chart showing a method for detecting the state of an earphone based on multiple sensors according to an embodiment of the present invention. Figure 1 As shown, the method of the present invention includes the following steps S210 to S230:

[0043] Step S210: acquiring first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by a first sound pickup sensor.

[0044] Step S220: Acquire second earphone state information according to the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor.

[0045] Step S230: output a final detection result of the headphone status based on the first headphone status information and the second headphone status information.

[0046] The above steps S210 and S220 are in parallel relationship and can be performed synchronously or asynchronously, and step S210 can also be performed after step S220. Step S230 integrates the headphone status information obtained in steps S210 and S220 and outputs the final detection result of the headphone status.

[0047] It can be seen that the method of the present invention combines the relationship between two types of signals to detect the status of the headphone. It not only utilizes the characteristic relationship between the signal input to the speaker and the signal picked up by the first sound pickup sensor, which can characterize the leakage of the headphone in low-frequency conditions and obtain the first headphone status information, but also utilizes the characteristic relationship between the signal picked up by the second sound pickup sensor and the signal picked up by the first sound pickup sensor, which can characterize the sound insulation effect of the headphone in medium and high frequency conditions and obtain the second headphone status information, and then fuses the first headphone status information and the second headphone status information to output the final detection result of the headphone status. Since the characteristic relationship between the two types of signals is combined when performing headphone status detection, it is possible to effectively divide the headphone status in a complex environment and improve the accuracy of headphone status detection.

[0048] In an acoustic system, the system transfer function (abbreviated as transfer function) characterizes the correlation component between two signals and is an optimal parameter. The relationship between the signals can be the amplitude-frequency characteristic or the phase-frequency characteristic of the transfer function. In the above steps S210 and S220, in order to obtain the headphone status information, it is necessary to estimate the system transfer function between the signals, or the correlation function. The estimation method of the transfer function and the correlation function is given below, and in order to avoid repeated description, the source audio signal sequence input to the speaker and the second audio signal sequence picked up by the second pickup sensor are combined and described:

[0049] (1) The signal of the current frame is obtained, one of which is the source audio signal sequence input to the speaker (or the second audio signal sequence picked up by the second sound pickup sensor), denoted by x = [x(0), x(1), ..., x(N-1)], and the other is the first audio signal sequence picked up by the first sound pickup sensor, denoted by y = [y(0), y(1), ..., y(N-1)], and the two signal sequences are high-pass filtered to filter out the influence of the DC signal.

[0050] (2) Windowing and frequency domain transformation: Add an analysis window to the two signals, such as a Hamming window (w = [w(0), w(1), ..., w(N-1)]), and perform Fourier transform to obtain frequency domain signals, which are denoted as X(k) and Y(k), respectively, as shown in the following formula:

[0051]

[0052]

[0053] Where N represents the number of Fourier transform points, n represents the signal sequence sample points, k represents the sequence number of the frequency bin, and bin represents the frequency interval or resolution of the frequency axis in the spectrum diagram.

[0054] (3) Calculation of auto-power spectrum and cross-power spectrum: The power spectrum can be estimated using the periodogram method. The calculation formula for the auto-power spectrum Pxx(k) of the first signal is as follows:

[0055]

[0056] The calculation formula of the second signal's autopower spectrum Pyy(k) is as follows:

[0057]

[0058] The cross power spectrum Pyx(k) of the two signals is calculated as follows:

[0059]

[0060] The * indicates conjugation.

[0061] (4) Determine whether the speaker or the first / second sound pickup sensor has a signal based on the size of the self-power spectrum. If the self-power spectrum is less than a certain threshold, such as -110dB, it is considered that the speaker or the first / second sound pickup sensor has no signal and an abnormality may have occurred. If there is a signal, proceed to the next step of calculation.

[0062] (5) Average power spectrum calculation: average smoothing is performed on the power spectrum within a period of time. For example, if the time length LenT = 30 frames, the average autopower spectrum PxxAve(k), PyyAve(k) and the average cross-power spectrum PyxAve(k) are calculated as follows:

[0063]

[0064]

[0065]

[0066] (6) Frequency domain transfer function H(k), calculated as follows:

[0067]

[0068] (7) Take the absolute value of the frequency domain transfer function and obtain the corresponding amplitude-frequency response |H(k)|:

[0069]

[0070] (8) The average phase of the frequency domain transfer function is calculated using the following method, where the imag function represents the imaginary part of a complex number, and the real function represents the real part of a complex number.

[0071]

[0072] (9) The related function is calculated using the following method, where abs means to obtain the absolute value and sqrt means to take the square root.

[0073]

[0074] It should be noted that the average correlation, average amplitude and average phase in a sub-band of a predetermined frequency range are obtained using the following calculation formula:

[0075]

[0076] Among them, subband represents the result of subband calculation, StartFreBin: subband starting frequency, EndFreBin: subband ending frequency, the frequency range of the subband can be divided into multiple continuous frequency bins (small frequency intervals) between StartFreBin and EndFreBin, and S represents the correlation, amplitude or phase calculated at each frequency bin.

[0077] Figure 3 The amplitude-frequency response of the transfer function between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor in different states is shown; Figure 4The phase-frequency response of the transfer function between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor in different states is shown; Figure 5 The amplitude-frequency response of the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor in different states is shown; Figure 6 The phase-frequency response of the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor in different states is shown. It can be seen that the transfer function between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor has different performances in different states, and the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor also has different performances in different states.

[0078] according to Figure 3 to Figure 6 It can be seen that according to the solution of this application, the state of the earphone can be divided into two categories: normal wearing state and abnormal state, and the normal wearing state is further divided into two situations: good coupling and slightly loose coupling, and the abnormal state is further divided into three situations: very loose coupling, open state, and abnormal howling. Among them, good coupling, slightly loose coupling, and very loose coupling are three different situations when wearing; the open state refers to the situation where the sound outlet of the earphone is completely exposed and basically unobstructed, such as the earphone is placed on the desktop; the abnormal howling situation refers to the situation where the earphone is in a very small cavity and abnormal howling may occur, such as holding the earphone tightly in the hand.

[0079] from Figure 3 It can be seen from the amplitude-frequency response that for the wearing conditions (good coupling, slightly loose coupling, very loose coupling), the looser the coupling between the earphone and the ear canal, the greater the attenuation of the low frequency (for example, below 400Hz); for the open state, relative to the wearing condition, in the mid-frequency band (for example, 300Hz-700Hz), the amplitude-frequency has obvious attenuation; for the abnormal howling condition, relative to the wearing condition, there is an obvious increase in the high frequency, especially above 1000Hz. Due to the obvious increase in the high frequency, there is a risk of howling.

[0080] If the source audio signal input to the speaker is a wide-band signal and the frequency information is sufficient, we can first distinguish whether it is an abnormal howling situation based on the high-frequency amplitude. For example, if there is a frequency band between 1000Hz-4000Hz with an amplitude exceeding the threshold TH1, then it is considered to be an abnormal howling situation; then we can distinguish whether it is an open state based on the intermediate frequency amplitude. For example, if the amplitude between 300Hz-700Hz is less than the threshold TH2, then it is considered to be an open state. Finally, we can judge the coupling situation when wearing it based on the low-frequency amplitude. For example, if it is less than a certain minimum threshold TH3, the coupling is very loose, if it is between the minimum threshold TH3 and the second lowest threshold TH4, the coupling is slightly loose, and if it is greater than the second lowest threshold TH4, the coupling is good.

[0081] In actual applications, the source audio signals input to the speaker may be various and may not be wide-band signals. If the source audio signals input to the speaker do not have enough mid-high frequency components, such as percussion music or single-frequency / multi-frequency with low frequency, and the main components are mainly at low frequencies, such as below 200Hz, then it is impossible to distinguish the open state and abnormal howling from the wearing state based on the amplitude alone. Figure 4 The low-frequency phase information can effectively improve the differentiation rate between abnormal howling and wearing conditions. For example, in the case of good coupling, the low-frequency amplitude is large and the phase is small; in the case of loose coupling, the amplitude is small and the phase is slightly small; and in the case of abnormal howling, the amplitude is large and the phase is also large. In addition, combined with Figure 5 and Figure 6 The amplitude and phase of the transfer function between the audio signals picked up by the first and second sound pickup sensors in the mid- and high-frequency range can further improve the differentiation rate between abnormal howling and wearing conditions. For example, if the coupling is very loose, there is no sound insulation effect in amplitude (small amplitude attenuation) and large attenuation in phase. However, in the case of abnormal howling, there is no sound insulation effect in amplitude, but there is almost no attenuation in phase from low frequency to mid-frequency (100Hz-1000Hz).

[0082] In practical applications, it is also possible that due to strong external noise, the correlation between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor is weak, and an effective transfer function cannot be estimated, and the headphone state cannot be determined. Then, the transfer function between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor can be used to roughly distinguish good coupling in the normal wearing state, slightly loose coupling, and very loose coupling in the abnormal state, open state and abnormal howling. For the wearing condition, the better the coupling, the better the sound insulation effect in the medium and high frequencies (such as around 1000hz), that is, the amount of external noise obtained by the first sound pickup sensor is significantly smaller than that of the second sound pickup sensor, and the external noise reaches the second sound pickup sensor first, and then reaches the first sound pickup sensor, which is manifested in the phase frequency as phase attenuation. However, there is usually no sound insulation effect in the amplitude frequency of the wearing very loose, open state and abnormal howling, or there may be an increase in a certain frequency band. The time for the external signal to reach the first sound pickup sensor and the second sound pickup sensor may be basically the same, which is manifested in the phase being close to 0.

[0083] The method of the present invention utilizes the relationship between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor, and then combines the relationship between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor, so as to improve the accuracy of various headphone state detection in complex scenarios, so that a series of controls or state adjustments can be made to the headphone according to different headphone states. For example, it can be known whether the headphone is worn on the ear, and a judgment or supplementary judgment of in-ear detection can be made; if it is in a wearing state, audio compensation can be made according to different coupling conditions, or a noise reduction filter can be selected or the noise reduction gain can be controlled to obtain a better noise reduction effect; if it is not in a wearing state, active noise reduction can be turned off to avoid howling, and audio playback can be turned off to save power consumption; or some reminders can be given to the user according to different states, such as the earplug is not suitable, or the headphone is abnormal, etc.

[0084] In addition, it should be noted that, considering that when the first sound pickup sensor fails, the first and second earphone status information cannot be obtained, resulting in the inability to perform the detection; when the first sound pickup sensor is normal, as long as one of the speaker or the second sound pickup sensor is normal, the detection can also be performed, although the detection accuracy will decrease. In the process of executing steps S210 to S230, once it is detected that the speaker has no input signal, the method of the present invention stops executing step S210, and directly outputs the final detection result of the earphone status according to the second earphone status information obtained in step S220; once it is detected that the first sound pickup sensor has no pickup signal, the execution of steps S210 and S220 is stopped, and a result prompt that the detection cannot be performed is output; once it is detected that the second sound pickup sensor has no pickup signal, the execution of step S220 is stopped, and the final detection result of the earphone status is directly output according to the first earphone status information obtained in step S210. Through such a setting, the detection efficiency and detection feasibility in the case of a speaker or sound pickup sensor failure can be balanced.

[0085] The above steps S210 to S230 are described in detail below.

[0086] Figure 7 A schematic diagram of a process of obtaining first earphone status information according to an embodiment of the present invention is shown. Figure 8 Shows the corresponding Figure 7 Schematic diagram of the logical judgment process. Figure 7 and Figure 8 The above step S210 "obtaining first earphone state information according to the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor" specifically includes steps S710 to S760:

[0087] S710: Calculate a first frequency domain transfer function and a first correlation function between a source audio signal and a first audio signal.

[0088] This step S710 can use the aforementioned steps (1) to (9) to calculate the first frequency domain transfer function and the first correlation function, which will not be described in detail here.

[0089] In addition, before step S710, it is necessary to determine whether the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor have signals in advance. If it is detected that there is no input signal to the speaker, a result prompt that the speaker may be faulty is output and subsequent detection is terminated; if it is detected that the first sound pickup sensor has no picked up signal, a result prompt that the first sound pickup sensor may be faulty is output and subsequent detection is terminated. Only when both the source audio signal and the first audio signal have signals, the subsequent detection is continued.

[0090] S720 , when the first frequency domain transfer function is stable, divide the first correlation function into sub-bands to obtain three sub-bands corresponding to low frequency, middle frequency and high frequency.

[0091] This step S720 can determine whether the transfer function is stable based on the size of the correlation function between the signals, or based on the variance of the transfer function. For example, the value of the first correlation function is higher than 0.8 and is considered stable, or the variance of the first frequency domain transfer function is less than 0.3 and is stable. If the first frequency domain transfer function is unstable, the default state (for example, good coupling) is output, or the determination result of the last first earphone state information is output. In the scheme of the present invention, the determination of the earphone wearing state is real-time and continuous. Therefore, when the current determination process cannot give a clear result, the last determination result can be selected as the default value.

[0092] When the first frequency domain transfer function is stable, the first correlation function is divided into multiple sub-bands, for example, into three sub-bands, sub-band 1: 100 Hz-200 Hz, corresponding to the low frequency sub-band; sub-band 2: 400 Hz-700 Hz, corresponding to the medium frequency sub-band; sub-band 3: 1000 Hz-3000 Hz, corresponding to the high frequency sub-band. The above steps (7) and (8) are used to calculate the average amplitude and average phase of the three sub-bands respectively.

[0093] S730: Determine whether the source audio signal input to the speaker is a wide-band signal or a narrow-band signal according to the correlation between the three sub-bands, or determine whether the external noise is very strong.

[0094] The step S730 specifically includes: when the average correlations of the three sub-bands are all high, determining that the source audio signal input to the speaker is a wide-band signal; when only the average correlation of the low-frequency sub-band is high, determining that the source audio signal input to the speaker is a narrow-band signal; when the average correlations of the three sub-bands are all low, determining that the external noise is very strong.

[0095] For example, if the average correlations of the three sub-bands are all high, such as the correlations of the three sub-bands are all higher than 0.8, then the source audio signal input to the speaker is considered to be a wideband signal. If only the average correlation of sub-band 1 is high, such as the average correlation of sub-band 1 is higher than 0.8, then the source audio signal input to the speaker is considered to be a narrowband signal. If the average correlations of the three sub-bands are all low, such as the correlations of the three sub-bands are all lower than 0.3, then the external noise is considered to be very strong.

[0096] S740: When it is determined that the source audio signal is a wideband signal, the headphone state is determined according to the amplitude-frequency characteristics of the three sub-bands corresponding to the low frequency, the middle frequency and the high frequency.

[0097] The step S740 specifically includes: determining whether the average amplitude of the first frequency domain transfer function corresponding to the high-frequency sub-band is greater than the first amplitude threshold, if so, determining it as an abnormal howling situation; otherwise, further determining whether the average amplitude of the first frequency domain transfer function corresponding to the intermediate frequency sub-band is less than the second amplitude threshold, if so, determining it as an open state; otherwise, determining it as a wearing situation, and further distinguishing it into three wearing situations: very loose coupling, slightly loose coupling and good coupling according to the size of the average amplitude of the first frequency domain transfer function corresponding to the low-frequency sub-band.

[0098] For example, if the source audio signal is a wide-band signal, the state is judged according to the average amplitude of each corresponding sub-band. If the average amplitude of sub-band 3 is higher than a certain threshold, such as higher than 0dB, it is considered to be an abnormal howling situation. Otherwise, the average amplitude of sub-band 2 is judged. If the amplitude of this frequency band is lower than a certain threshold, such as -10dB, it is considered to be an open state. Otherwise, it is a wearing state. Then the coupling situation is judged according to the average amplitude of sub-band 1. If the amplitude is less than -10dB, it is considered to be very loosely coupled, such as the earphones are slightly hung on the ears. If the amplitude is greater than or equal to -10dB and less than -3dB, it is considered to be slightly loosely coupled. If the amplitude is greater than or equal to -3dB, it is considered to be well coupled.

[0099] S750: When it is determined that the source audio signal is a narrow-band signal, determine the state of the earphone according to the amplitude-frequency characteristic and the phase-frequency characteristic of the first frequency domain transfer function corresponding to the low-frequency sub-band.

[0100] The step S750 specifically includes: obtaining the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band; if the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset first amplitude range and first phase range, it is determined to be a normal wearing state with good coupling; if the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset second amplitude range and second phase range, it is determined to be a normal wearing state with slightly loose coupling; otherwise, it is determined to be an abnormal state.

[0101] For example, if the source audio signal is a narrowband signal, the state is judged based on the average amplitude and average phase of sub-band 1. For example, if the average amplitude is greater than or equal to -3dB and less than 5dB, and the phase is less than 3 degrees, it is considered to be a well-coupled wearing condition; if the amplitude is greater than or equal to -10dB and less than -3dB, and the phase is greater than or equal to 3 degrees and less than 23 degrees, it is considered to be a slightly loosely coupled wearing condition; if these two conditions are not met, it is considered to be an abnormal state.

[0102] S760, when it is determined that the external noise is very strong, a result prompt of no valid state is output.

[0103] When the external noise is very strong, the correlation between the source audio signal and the first audio signal is weak, and an effective transfer function cannot be estimated, and thus the wearing state cannot be determined. In this case, a result prompt of no valid state is output.

[0104] Fig. 9 A schematic diagram of a process for obtaining second earphone status information according to an embodiment of the present invention is shown. Fig.10 Shows the corresponding Fig. 9 Schematic diagram of the logical judgment process. Fig. 9 and Fig.10 The above step S220 "obtaining the second earphone state information according to the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor" specifically includes steps S910 to S930:

[0105] S910: Calculate a second frequency domain transfer function between the second audio signal and the first audio signal.

[0106] This step S910 can use the aforementioned steps (1) to (9) to calculate the second frequency domain transfer function, which will not be described in detail here.

[0107] In addition, before step S910, it is necessary to judge whether the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor have signals in advance. If it is detected that the second sound pickup sensor does not pick up signals, the result prompt that the second sound pickup sensor may be faulty is output and the subsequent detection is terminated; if it is detected that the first sound pickup sensor does not pick up signals, the result prompt that the first sound pickup sensor may be faulty is output and the subsequent detection is terminated. Only when both the second audio signal and the first audio signal have signals, the subsequent detection is continued.

[0108] S920 , when the second frequency domain transfer function is stable, obtain an average amplitude and an average phase of the second frequency domain transfer function corresponding to the middle and high frequency sub-bands.

[0109] In step S920, whether the transfer function is stable can be determined based on the variance of the transfer function. For example, if the variance of the second frequency domain transfer function is less than 0.3, it is considered stable. If the second frequency domain transfer function is unstable, a default state (for example, good coupling) is output or the determination result of the last second earphone state information is output.

[0110] When the second frequency domain transfer function is stable, obtain the mid-high frequency sub-bands, for example, obtain sub-band 4: 600 Hz-900 Hz, and calculate the average amplitude and average phase of sub-band 4 using the aforementioned steps (7) and (8).

[0111] S930, if the average amplitude and average phase of the second frequency domain transfer function corresponding to the mid-high frequency sub-band are within the respective preset ranges of amplitude and phase, it is determined to be a normal wearing state, and is further distinguished into a normal wearing state with good coupling and a normal wearing state with slightly loose coupling according to the average amplitude of the mid-high frequency sub-band, otherwise it is determined to be an abnormal state.

[0112] For example, if the average amplitude and average phase of sub-band 4 meet the preset wearing conditions, such as the average amplitude of sub-band 4 is greater than -12dB and less than -2dB, and the average phase of sub-band 4 is greater than -100 degrees and less than -18 degrees, then it is considered to be in a normal wearing state, otherwise it is an abnormal state. If it is in a normal wearing state, the coupling condition is further judged according to the average amplitude of sub-band 4. If the average amplitude of sub-band 4 is less than -6dB, then it is considered to be well coupled, otherwise it is considered to be slightly loosely coupled.

[0113] Fig.11 A schematic diagram of a process of fusing and outputting two types of headphone status information according to an embodiment of the present invention is shown. Fig.12 Shows the corresponding Fig.11 Schematic diagram of the logical judgment process. Fig.11 and Fig.12 The above step S230 "outputting the final detection result of the headphone status based on the first headphone status information and the second headphone status information" specifically includes steps S110 to S130:

[0114] Step S110: when the first earphone status information is in an invalid state, directly output the final detection result of the earphone status according to the second earphone status information.

[0115] The corresponding scenario of step S110 is: the sound of the source audio signal input to the speaker is small or silent, and the external environment is noisy. The signal emitted by the speaker picked up by the first sound pickup sensor is seriously polluted by noise. The main component of the first audio signal is external noise. It is impossible to effectively obtain the characteristic relationship between the source audio signal and the first audio signal, and thus it is impossible to obtain the first earphone status information. At this time, the characteristic relationship between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor can be effectively obtained, and the second earphone status information can be effectively obtained based on this. Therefore, step S110 first determines whether the first earphone status information is in an invalid state. If so, the second earphone status information is directly output, otherwise it goes to step S120.

[0116] Step S120: when the first earphone state information is a result obtained based on the broadband signal, directly output the final detection result of the earphone state according to the first earphone state information.

[0117] The corresponding scenario of step S120 is that in a quiet environment, the first sound pickup sensor mainly picks up the signal emitted by the speaker, and it is easier to obtain the characteristic relationship between the source audio signal and the first audio signal, so as to judge the status of the earphone. Therefore, when the first earphone status information has a valid status output and is a result obtained based on a wideband signal, the first earphone status information is directly output, otherwise it goes to step S130.

[0118] Step S130, when the first earphone status information is a result obtained based on a narrowband signal, if the obtained first earphone status information and second earphone status information are both in a normal wearing state, the first earphone status information is used as the final detection result of the earphone status, otherwise the final detection result of the earphone status is output as an abnormal state.

[0119] The corresponding scenario of step S130 is that when the speaker signal information is insufficient, such as only low frequency, there will be false detection in distinguishing the headphone status. At this time, the headphone status is judged in combination with the characteristic relationship between the signals picked up by the second sound pickup sensor and the first sound pickup sensor, which can improve the correct detection rate. And when the first headphone status information and the second headphone status information are both in the normal wearing state, the coupling state given by the first headphone status information is output first; and when the first headphone status information and the second headphone status information are in different states, for example, the first headphone status information is in the open state and the second headphone status information is in the abnormal state, in order to avoid giving the wrong headphone status, the final detection result of the output headphone status is set to be the abnormal state.

[0120] In summary, the headphone state detection method based on multiple sensors of the present invention can improve the accuracy of distinction in complex scenarios by distinguishing the headphone states according to multiple sensors. According to the characteristic relationship between the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor, the headphone multi-state distinction is performed, including wearing conditions of good coupling, slightly loose coupling and very loose coupling, open state, and abnormal howling conditions, etc., and further combined with the characteristic relationship between the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor, the headphone multi-state distinction is performed, including normal wearing conditions of good coupling and slightly loose coupling and abnormal conditions, etc., to improve the accuracy of state distinction in complex scenarios, and then the headphone function can be effectively controlled and adjusted according to product requirements using the state output results.

[0121] The present invention also provides a headphone status detection device based on multiple sensors, which is based on the same technical concept as the aforementioned headphone status detection method based on multiple sensors. Fig.13 FIG. 1 is a schematic diagram showing a structure of a headphone state detection device based on multiple sensors according to an embodiment of the present invention. Fig.13 As shown, the device of the present invention comprises:

[0122] A first state acquisition module 131, configured to acquire first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by a first sound pickup sensor;

[0123] A second state acquisition module 132, configured to acquire second earphone state information according to a second audio signal picked up by the second sound pickup sensor and a first audio signal picked up by the first sound pickup sensor;

[0124] The state fusion output module 133 is used to output a final detection result of the headphone state based on the first headphone state information and the second headphone state information.

[0125] Fig.14 FIG. 1 is a schematic diagram showing the structure of a first state acquisition module according to an embodiment of the present invention. Fig.14 As shown, the first state acquisition module 131 includes:

[0126] A first calculation unit 1311 is used to calculate a first frequency domain transfer function and a first correlation function between the source audio signal and the first audio signal;

[0127] A sub-band division unit 1312 is used to divide the first correlation function into sub-bands when the first frequency domain transfer function is stable, so as to obtain three sub-bands corresponding to low frequency, medium frequency and high frequency;

[0128] The scene determination unit 1313 is used to determine whether the source audio signal input to the speaker is a wide-band signal or a narrow-band signal, or to determine whether the external noise is very strong, according to the correlation between the three sub-bands.

[0129] The broadband scene determination unit 1314 is used to determine the headphone state according to the amplitude-frequency characteristics of the three sub-bands corresponding to the low frequency, the middle frequency and the high frequency when determining that the source audio signal is a broadband signal;

[0130] The narrowband scene determination unit 1315 is used to determine the headphone state according to the amplitude-frequency characteristics and phase-frequency characteristics of the corresponding low-frequency sub-band when determining that the source audio signal is a narrowband signal;

[0131] The noise scene output unit 1316 is used to output a result prompt of no valid state when it is determined that the external noise is very strong.

[0132] In one embodiment, the scene judgment unit 1313 is specifically used to: when the average correlations of the three sub-bands are all high, determine that the source audio signal input to the speaker is a wide-band signal; when only the average correlation of the low-frequency sub-band is high, determine that the source audio signal input to the speaker is a narrow-band signal; when the average correlations of the three sub-bands are all low, determine that the external noise is very strong.

[0133] In one embodiment, the broadband scene judgment unit 1314 is specifically used to: judge whether the average amplitude of the first frequency domain transfer function corresponding to the high-frequency sub-band is greater than the first amplitude threshold, if so, it is determined as an abnormal howling situation; otherwise, further judge whether the average amplitude of the first frequency domain transfer function corresponding to the intermediate frequency sub-band is less than the second amplitude threshold, if so, it is determined to be an open state; otherwise, it is determined to be a wearing situation, and further distinguished into three wearing situations: very loose coupling, slightly loose coupling and good coupling according to the size of the average amplitude of the first frequency domain transfer function corresponding to the low-frequency sub-band.

[0134] In one embodiment, the narrowband scene judgment unit 1315 is specifically used to: obtain the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band; if the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset first amplitude range and first phase range, it is determined to be a normal wearing state with good coupling; if the average amplitude and average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset second amplitude range and second phase range, it is determined to be a normal wearing state with slightly loose coupling; otherwise, it is determined to be an abnormal state.

[0135] Fig.15 FIG. 1 is a schematic diagram showing the structure of a second state acquisition module according to an embodiment of the present invention. Fig.15 As shown, the second state acquisition module 132 includes:

[0136] A second calculation unit 1321, configured to calculate a second frequency domain transfer function between the second audio signal and the first audio signal;

[0137] A mid-high frequency sub-band acquisition unit 1322 is used to acquire an average amplitude and an average phase of the second frequency domain transfer function corresponding to the mid-high frequency sub-band when the second frequency domain transfer function is stable;

[0138] The mid-high frequency sub-band judgment unit 1323 is used to judge it as a normal wearing state if the average amplitude and average phase of the second frequency domain transfer function corresponding to the mid-high frequency sub-band are within the respective preset ranges of amplitude and phase, and further distinguish it into a normal wearing state with good coupling and a normal wearing state with slightly loose coupling according to the average amplitude of the mid-high frequency sub-band; otherwise, it is judged as an abnormal state.

[0139] Fig.16 FIG. 1 is a schematic diagram showing the structure of a state fusion output module according to an embodiment of the present invention. Fig.16 As shown, the state fusion output module 133 includes:

[0140] The first scene fusion output unit 1331 is used to directly output a final detection result of the headphone state according to the second headphone state information when the first headphone state information is in an invalid state;

[0141] The second scene fusion output unit 1332 is used to output a final detection result of the headphone state directly according to the first headphone state information when the first headphone state information is a result obtained based on the broadband signal;

[0142] The third scene fusion unit 1333 is used to output the coupling state of normal wearing of the earphone according to the first earphone state information when the first earphone state information is a result obtained based on a narrowband signal, if the obtained first earphone state information and second earphone state information are both normal wearing states, otherwise the final detection result of the earphone state is output as an abnormal state.

[0143] The implementation process of each module or unit in the multi-sensor based earphone state detection device of the present invention can be referred to the aforementioned method embodiment, which will not be described in detail here.

[0144] The present invention also provides a headset, which is of the same technical concept as the aforementioned headset status detection method and device based on multiple sensors. Fig.17 FIG. 1 is a schematic diagram showing the structure of an earphone according to an embodiment of the present invention. Fig.17The earphone provided by the present invention has a memory and a processor, and also has a speaker located in the ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal; wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the aforementioned multi-sensor based earphone status detection method, which will not be repeated here.

[0145] At the hardware level, the headset may also include wireless communication modules such as Bluetooth and WIFI, etc. The memory, processor, speaker, first sound pickup sensor, second sound pickup sensor, wireless communication module, etc. may be interconnected through an internal bus, which may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.17 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0146] The present invention also proposes a computer-readable storage medium, which stores one or more computer programs. When the one or more computer programs are executed by a processor, the aforementioned multi-sensor based earphone status detection method is implemented, which will not be repeated here.

[0147] It will be appreciated by those skilled in the art that the various embodiments of the present invention may be provided as methods, devices, headphones or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware embodiments.

[0148] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0149] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting the state of an earphone based on multiple sensors, wherein the earphone comprises a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal, wherein: The method comprises: Acquire first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by the first sound pickup sensor; Acquire second earphone state information according to the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor; Based on the first earphone state information and the second earphone state information, a final detection result of the earphone state is output.

2. The method according to claim 1, characterized in that: The outputting a final detection result of the headphone state based on the first headphone state information and the second headphone state information includes: When the first earphone state information is in an invalid state, directly outputting a final detection result of the earphone state according to the second earphone state information; When the first earphone state information is a result obtained based on the broadband signal, directly outputting a final detection result of the earphone state according to the first earphone state information; When the first earphone status information is obtained based on a narrowband signal, if the first earphone status information and the second earphone status information obtained are both normal wearing states, the final detection result of the earphone status is output according to the first earphone status information, otherwise the final detection result of the earphone status is output as an abnormal state.

3. The method according to claim 1, characterized in that: The obtaining the first earphone state information according to the source audio signal input to the speaker and the first audio signal picked up by the first sound pickup sensor includes: calculating a first frequency domain transfer function and a first correlation function between the source audio signal and the first audio signal; When the first frequency domain transfer function is stable, the first correlation function is divided into sub-bands to obtain three sub-bands corresponding to low frequency, medium frequency and high frequency; According to the correlation of the three sub-bands, determining whether the source audio signal input to the speaker is a wide-band signal or a narrow-band signal, or determining whether the external noise is very strong; When it is determined that the source audio signal is a wideband signal, judging the state of the earphone according to the amplitude-frequency characteristics of the three sub-bands corresponding to the low frequency, the medium frequency and the high frequency; When it is determined that the source audio signal is a narrowband signal, judging the state of the earphone according to the amplitude-frequency characteristics and the phase-frequency characteristics of the corresponding low-frequency sub-band; When it is determined that the external noise is very strong, a result prompt indicating that there is no valid state is output.

4. The method according to claim 3, characterized in that: The step of judging whether the source audio signal input to the speaker is a wideband signal or a narrowband signal according to the correlation of the three sub-bands, or judging whether the external noise is very strong, includes: When the average correlations of the three sub-bands are all high, determining that the source audio signal input to the speaker is a wide-band signal; When only the average correlation of the low-frequency sub-band is high, determining that the source audio signal input to the speaker is a narrow-band signal; When the average correlations of the three sub-bands are all low, it is determined that the external noise is very strong.

5. The method according to claim 3, characterized in that: When determining that the source audio signal is a wideband signal, judging the state of the earphone according to the amplitude-frequency characteristics of the three sub-bands corresponding to the low frequency, the intermediate frequency and the high frequency, comprises: Determine whether the average amplitude of the first frequency domain transfer function corresponding to the high frequency sub-band is greater than a first amplitude threshold, and if so, determine that it is an abnormal howling situation; otherwise, It is further determined whether the average amplitude of the first frequency domain transfer function corresponding to the intermediate frequency sub-band is less than a second amplitude threshold, and if so, it is determined to be an open state; otherwise, It is determined as a wearing condition, and further distinguished into three wearing conditions: very loose coupling, slightly loose coupling, and good coupling according to the average amplitude of the first frequency domain transfer function corresponding to the low-frequency sub-band.

6. The method according to claim 3, characterized in that: When determining that the source audio signal is a narrowband signal, determining the state of the earphone according to the amplitude-frequency characteristics and the phase-frequency characteristics of the corresponding low-frequency sub-band includes: Obtaining an average amplitude and an average phase of the first frequency domain transfer function corresponding to the low frequency sub-band; If the average amplitude and the average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset first amplitude range and first phase range, it is determined to be a normal wearing state with good coupling; If the average amplitude and the average phase of the first frequency domain transfer function corresponding to the low-frequency sub-band are both within the preset second amplitude range and the second phase range, it is determined to be a normal wearing state with slightly loose coupling; Anything else is considered an abnormal state.

7. The method according to claim 1, characterized in that: The acquiring the second earphone state information according to the second audio signal picked up by the second sound pickup sensor and the first audio signal picked up by the first sound pickup sensor comprises: calculating a second frequency domain transfer function between the second audio signal and the first audio signal; When the second frequency domain transfer function is stable, obtaining an average amplitude and an average phase of the second frequency domain transfer function corresponding to the middle and high frequency sub-bands; If the average amplitude and average phase of the second frequency domain transfer function corresponding to the mid-high frequency sub-band are within the respective preset ranges of amplitude and phase, it is determined to be a normal wearing state, and is further distinguished into a normal wearing state with good coupling and a normal wearing state with slightly loose coupling according to the average amplitude of the mid-high frequency sub-band; otherwise it is determined to be an abnormal state.

8. A multi-sensor based earphone state detection device, the earphone having a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal, characterized in that: The device comprises: A first state acquisition module, configured to acquire first earphone state information according to a source audio signal input to the speaker and a first audio signal picked up by the first sound pickup sensor; A second state acquisition module, used for acquiring second earphone state information according to a second audio signal picked up by the second sound pickup sensor and a first audio signal picked up by the first sound pickup sensor; The state fusion output module is used to output a final detection result of the headphone state based on the first headphone state information and the second headphone state information.

9. An earphone, the earphone having a memory and a processor, and further having a speaker located in an ear canal, a first sound pickup sensor located in the ear canal and arranged near the speaker, and a second sound pickup sensor located outside the ear canal; The memory stores a computer program, which is loaded and executed by the processor to implement the multi-sensor based earphone status detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing one or more computer programs, wherein the one or more computer programs, when executed by a processor, implement the multi-sensor based earphone status detection method according to any one of claims 1 to 8.

Images