Headphone wearing detection method, device, headphone and storage medium

Abstract

The embodiments of the present application disclose a headphone wearing detection method, device, headphone and storage medium, belonging to the field of headphone technology. The method includes: playing a first audio signal through a speaker, and obtaining a second audio signal collected by a feedforward microphone, and a third audio signal collected by a feedback microphone; determining the cross-correlation value between the first audio signal, the second audio signal and the third audio signal, the cross-correlation value is used to indicate the degree of correlation between the audio signals; determining the headphone state of the headphone based on the cross-correlation value, the headphone state includes a wearing state and a not-wearing state. Using the method provided in the embodiments of the present application, the headphone state is determined by detecting the correlation between audio signals, the detection function is stable, and the detection accuracy can be improved, and the probability of misjudgment can be reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of earphone technology, and in particular to an earphone wearing detection method, device, earphone, and storage medium. Background Art

[0002] Headphone wearing detection refers to detecting the current wearing status of the headphones. If the headphones are not in the wearing status, the music playback and active noise cancellation (ANC) functions of the headphones can be turned off to reduce power consumption.

[0003] In the related art, by collecting parameters such as earphone movement data, wearing pressure data, earphone surface temperature and working power, the parameters are compared with corresponding thresholds, and then the wearing status of the earphone is determined based on the comparison results.

[0004] However, in the related art, sensors may be used to collect data. If the sensors are touched by mistake, the judgment results will be inaccurate, and the use of sensors will increase the hardware cost. In addition, the collection of information such as temperature is easily affected by environmental differences, which in turn affects the accuracy of the judgment results. Summary of the invention

[0005] The embodiments of the present application provide a method, device, headset and storage medium for detecting wearing of a headset. The technical solution is as follows:

[0006] On the one hand, an embodiment of the present application provides a method for detecting wearing of an earphone, the method being used for an earphone provided with a feedforward microphone, a feedback microphone, and a speaker, the method comprising:

[0007] Playing a first audio signal through the speaker, and acquiring a second audio signal collected by the feedforward microphone and a third audio signal collected by the feedback microphone;

[0008] Determine a cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal, wherein the cross-correlation value is used to indicate a correlation degree between the audio signals;

[0009] A headphone state of the headphone is determined based on the cross-correlation value, where the headphone state includes a wearing state and a not-wearing state.

[0010] On the other hand, an embodiment of the present application provides an earphone wearing detection device, the device comprising:

[0011] A first acquisition module, used for playing a first audio signal through the speaker, and acquiring a second audio signal collected by the feedforward microphone, and a third audio signal collected by the feedback microphone;

[0012] A first determining module, configured to determine a cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal, wherein the cross-correlation value is used to indicate a correlation degree between the audio signals;

[0013] The second determining module is used to determine the headphone state of the headphone based on the cross-correlation value, where the headphone state includes a wearing state and a not-wearing state.

[0014] On the other hand, an embodiment of the present application provides an earphone, which includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the earphone wearing detection method as described in the above aspect.

[0015] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the headphone wearing detection method as described in the above aspects.

[0016] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the headphone wearing detection method provided in the above aspect.

[0017] The technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0018] By adopting the method provided in the embodiment of the present application, an audio signal is played through a speaker, and the audio signal is collected by using a feedforward microphone and a feedback microphone arranged at different positions of the earphone, so that the correlation degree between the two signals in the three audio signals is obtained according to the cross-correlation values ​​between the two signals in the three audio signals, and then whether the earphone is in a wearing state is judged according to the correlation degree between the signals, thereby improving the accuracy of earphone wearing detection, and the correlation between the detection signals is not affected by environmental factors or false touch of the sensor, the detection function is stable, and it helps to reduce the probability of misjudgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A structural block diagram of a headset provided by an exemplary embodiment of the present application is shown;

[0020] Figure 2 A schematic diagram showing the distribution of earphone components provided by an exemplary embodiment of the present application is shown;

[0021] Figure 3A method flow chart of a headphone wearing detection method provided by another exemplary embodiment of the present application is shown;

[0022] Figure 4 A method flow chart of a headphone wearing detection method provided by another exemplary embodiment of the present application is shown;

[0023] Figure 5 A method flow chart of a headphone wearing detection method provided by another exemplary embodiment of the present application is shown;

[0024] Figure 6 A schematic diagram showing a headphone wearing detection process provided by another exemplary embodiment of the present application is shown;

[0025] Figure 7 A structural block diagram of an earphone wearing detection device provided in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0027] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0028] Please refer to Figure 1 , which shows a structural block diagram of an earphone provided by an exemplary embodiment of the present application. The earphone in the present application may include one or more of the following components: a processor 101, a memory 102, a feedforward microphone 103, a feedback microphone 104, and a speaker 105, etc.

[0029] The processor 101 may include one or more processing cores. The processor 101 uses various interfaces and lines to connect the various parts of the entire headset, and executes various functions of the headset and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 102, and calling data stored in the memory 102. For example, it can be used to implement the headset noise reduction function and wearing detection function. Optionally, the processor 101 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA).

[0030] The memory 102 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 102 includes a non-transitory computer-readable storage medium. The memory 102 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 102 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a noise reduction function, an audio playback function, a wearing detection function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data created according to the use of the headset, etc.

[0031] The feedforward microphone 103 is used to collect ambient sound outside the earphone and input the audio signal to the processor 101 for processing. The feedforward microphone 103 can be arranged outside the earphone away from the speaker. The feedforward microphone type can be an array microphone or an omnidirectional collection microphone.

[0032] The feedback microphone 104 is used to collect audio near the speaker, including the audio played by the speaker and the ambient sound, and can input the collected audio signal to the processor 101 for processing. The feedback microphone 104 can be set at the inner side of the earphone near the speaker. And the feedback microphone type can be an array microphone or an omnidirectional collection microphone.

[0033] The speaker 105 is used to convert the electrical signal from the processor 101 into sound waves. The speaker can be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans to achieve the headphone noise reduction function, etc.

[0034] In addition, those skilled in the art will appreciate that the structure of the earphones shown in the above figures does not constitute a limitation on the earphone device, and the earphone device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, it may include a power supply, a sensor, a Bluetooth module, and a display component, etc., which will not be described in detail here.

[0035] Hybrid active noise reduction headphones use ANC technology to reduce noise, such as Figure 2As shown, the speaker 203 emits an anti-phase sound wave with respect to the noise collected by the feedforward microphone 201 and the feedback microphone 202 to neutralize the noise, thereby realizing the noise reduction function. As the size of headphones becomes smaller and smaller, the power consumption requirement becomes higher and higher. Therefore, it is necessary to detect whether the headphones are in the wearing state through the headphone wearing detection. If they are not in the wearing state, the music playback function and the ANC function can be stopped to reduce power consumption and extend the battery life of the headphones.

[0036] In the related technologies, sensors are often used for detection, such as acceleration sensors, pressure sensors, etc., to collect headphone motion data or wearing pressure data, and then judge the wearing condition of the headphone. However, the use of sensors is prone to false touches of sensors, resulting in misjudgment, and the use of sensors increases certain hardware costs. Alternatively, there are also methods in the related technologies that judge the wearing condition of the headphone by detecting changes in headphone temperature, working power, etc., but these methods are easily affected by environmental differences, resulting in misjudgment of the headphone status.

[0037] An embodiment of the present application provides a headphone wearing detection method, which determines whether the headphone is in a wearing state based on the correlation between the sound source played by the speaker and the audio signal collected by the feedforward microphone and the feedback microphone. The detection function is stable, which can reduce the probability of misjudgment caused by false touch of the sensor or environmental factors, and improve the accuracy of headphone wearing detection.

[0038] Please refer to Figure 3 , which shows a method flow chart of a headphone wearing detection method provided by an exemplary embodiment of the present application, and the method may include the following steps.

[0039] Step 301: Play a first audio signal through a speaker, and obtain a second audio signal collected by a feedforward microphone and a third audio signal collected by a feedback microphone.

[0040] In a possible implementation manner, the method provided in the embodiments of the present application can be used as follows: Figure 2 The hybrid active noise reduction headset shown plays a first audio signal through a speaker, collects a second audio signal through a feedforward microphone, and collects a third audio signal through a feedback microphone. The second audio signal refers to the ambient noise signal around the headset, and the third audio signal refers to the audio signal near the speaker, which includes the audio signal played by the speaker and the noise signal near the speaker. In the embodiment of the present application, the first audio signal, the second audio signal and the third audio signal refer to audio signals in the same time period.

[0041] Optionally, the method provided in the embodiment of the present application can also be used for headphones other than the above-mentioned hybrid active noise reduction headphones. If the headphones are non-ANC headphones, the second audio signal can be collected by an external microphone that is similar in position and shape to the feedforward microphone, and the third audio signal can be collected by an internal microphone that is similar in position and shape to the feedback microphone; if the headphones are ANC-type single feedback headphones (i.e., headphones that only include a feedback microphone), the second audio signal can be collected by the call microphone (external microphone).

[0042] Step 302: determine the cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal. The cross-correlation value is used to indicate the degree of correlation between the audio signals.

[0043] In a possible implementation, the headphone state is determined using the correlation between the first audio signal, the second audio signal, and the third audio signal. The correlation can be represented by a cross-correlation value between the signals, wherein the cross-correlation value is positively correlated with the correlation between the audio signals. When the cross-correlation value between the audio signals is larger, it indicates that the correlation between the audio signals is higher, that is, the two audio signals are more similar.

[0044] Step 303: determine the headphone state of the headphone based on the cross-correlation value, where the headphone state includes a wearing state and a not-wearing state.

[0045] In a possible implementation, the headphone state of the headphone can be determined by any of the cross-correlation values ​​between the two signals, for example, the headphone state of the headphone can be determined by the cross-correlation value between the second audio signal and the third audio signal, or the headphone state of the headphone can be determined by the cross-correlation value between the second audio signal and the third audio signal. However, this method is only based on the correlation between the two audio signals, which is prone to misjudgment. Therefore, in another possible implementation, after determining the cross-correlation value between the two signals, the headphone state of the headphone can be determined by the size of the three cross-correlation values ​​to improve the detection accuracy.

[0046] Optionally, in an embodiment of the present application, the headphone state refers to a wearing state and a not-wearing state. When the cross-correlation value between the first audio signal and the second audio signal, the cross-correlation value between the first audio signal and the third audio signal, and the cross-correlation value between the second audio signal and the third audio signal all meet certain conditions, it is determined that the headphone is in a wearing state, and when any of the cross-correlation values ​​does not meet the conditions, it is determined that the headphone is in a not-wearing state. Compared with the method used in the related art, it can avoid misjudgment of the headphone state due to false touch of the sensor or misjudgment of the headphone state due to interference from environmental factors.

[0047] To summarize, in the embodiment of the present application, an audio signal is played through a speaker, and audio signals are collected by using a feedforward microphone and a feedback microphone arranged at different positions of the earphone, so that the correlation degree between each two of the three audio signals is obtained according to the cross-correlation values ​​between each two of the three audio signals, and then whether the earphone is in a wearing state is judged according to the correlation degree between the signals, thereby improving the accuracy of earphone wearing detection, and the correlation between the detection signals is not affected by environmental factors or false touch of the sensor, and the detection function is stable, which helps to reduce the probability of misjudgment.

[0048] In a possible implementation, when determining the state of the earphone, a threshold may be set for the cross-correlation value between different signals, and then the state of the earphone is determined according to the cross-correlation value between different signals and its corresponding threshold, which will be described below with an exemplary embodiment.

[0049] Please refer to Figure 4 , which shows a method flow chart of a headphone wearing detection method provided by an exemplary embodiment of the present application, and the method may include the following steps.

[0050] Step 401: play a first audio signal through a speaker, and obtain a second audio signal collected by a feedforward microphone and a third audio signal collected by a feedback microphone.

[0051] The implementation of this step may refer to step 301 and will not be described in detail in this embodiment.

[0052] Step 402: determine a first mutual correlation value between the second audio signal and the third audio signal, a second mutual correlation value between the first audio signal and the second audio signal, and a third mutual correlation value between the first audio signal and the third audio signal.

[0053] In one possible implementation, a first cross-correlation value between a second audio signal collected by a feedforward microphone and a third audio signal collected by a feedback microphone, a second cross-correlation value between a first audio signal played by a speaker and the second audio signal collected by the feedforward microphone, and a third cross-correlation value between the first audio signal played by the speaker and the third audio signal collected by the feedback microphone are determined, and the first cross-correlation value, the second cross-correlation value, and the third cross-correlation value are used to jointly determine whether the earphone is in a wearing state.

[0054] Step 403: Determine the earphone state based on the first mutual correlation value, the second mutual correlation value, the third mutual correlation value, and thresholds corresponding to the mutual correlation values.

[0055] Since the second audio signal refers to the ambient noise signal around the headset, and the third audio signal refers to the audio signal near the speaker, when the correlation between the second audio signal and the third audio signal is smaller, that is, the first mutual correlation value is smaller, it indicates that the headset may be in a wearing state; when the correlation between the first audio signal and the third audio signal is smaller, that is, the second mutual correlation value is smaller, it indicates that the headset may be in a wearing state; when the correlation between the first audio signal and the third audio signal, that is, the third mutual correlation value, is larger, it indicates that the headset may be in a wearing state. In a possible implementation, after determining the first mutual correlation value, the second mutual correlation value, and the third mutual correlation value, the magnitude of the mutual correlation value can be determined according to the threshold value corresponding to each mutual correlation value, and then the headset state can be determined.

[0056] In a possible implementation, in response to the first mutual correlation value being less than a first threshold, the second mutual correlation value being less than a second threshold, and the third mutual correlation value being greater than a third threshold, it is determined that the earphone is in a wearing state.

[0057] Optionally, the first threshold, the second threshold, and the third threshold may be the same or different in size, and may be determined according to the actual detected audio signal. For example, the first threshold, the second threshold, and the third threshold may be the same value, or the first threshold and the second threshold may be the same in size, and the third threshold may be greater than the first threshold and the second threshold, etc. The embodiment of the present application does not limit the size of the first threshold, the second threshold, and the third threshold.

[0058] In a possible implementation, in response to the first mutual correlation value being greater than a first threshold, the second mutual correlation value being greater than a second threshold, or the third mutual correlation value being less than a third threshold, it is determined that the earphone is in an unworn state.

[0059] That is, when any one of the first mutual correlation value is greater than the first threshold, the second mutual correlation value is greater than the second threshold, and the third mutual correlation value is less than the third threshold exists, it can be determined that the earphone is in the unworn state.

[0060] Step 404: In response to the earphone being in a wearing state and the third mutual correlation value being greater than a fifth threshold, it is determined that the earphone is in a normal wearing state.

[0061] When it is determined that the earphone is in a wearing state, it is possible that the earphone is not worn well, and the tightness of the earphone wearing has a greater impact on the active noise reduction function and the low-frequency sound quality. Therefore, in a possible implementation, after determining that the earphone is in a wearing state, it is also possible to determine whether the earphone is in a normal wearing state through the first mutual correlation value, the second mutual correlation value, and the third mutual correlation value, that is, to detect the tightness of the earphone wearing. When the first mutual correlation value and the second mutual correlation value are smaller, and the third mutual correlation value is larger, it indicates that the earphone is worn well and is in a normal wearing state.

[0062] However, judging the tightness of earphone wearing by three cross-correlation values ​​requires a large amount of calculation. In order to further simplify the detection process and improve the detection efficiency, in another possible implementation, the wearing condition of the earphone can be judged only by the third cross-correlation value. When the third cross-correlation value is large, it indicates that the first audio signal played by the speaker is similar to the third audio signal collected by the feedback microphone. When the first audio signal is similar to the third audio signal, it indicates that the noise in the ear canal collected by the feedback microphone is small, and the earphone is in good wearing condition. Therefore, in a possible implementation, when the third cross-correlation value is greater than the fifth threshold, it can be determined that the earphone is in a normal wearing state. Optionally, the fifth threshold is greater than the third threshold.

[0063] Step 405: In response to the earphone being in a wearing state and the third mutual correlation value being less than a fifth threshold, it is determined that the earphone is in an abnormal wearing state, and a prompt is given.

[0064] In a possible implementation, when the third mutual correlation value is less than the fifth threshold, it indicates that there are many differences between the first audio signal played by the speaker and the third audio signal collected by the feedback microphone, the noise in the ear canal collected by the feedback microphone is large, and the earphone wearing condition is abnormal. Therefore, in a possible implementation, when the third mutual correlation value is less than the fifth threshold, it can be determined that the earphone is in an abnormal wearing state.

[0065] Optionally, when the headset is detected to be in an abnormal wearing state, a corresponding prompt may be given. When giving a prompt, a display component of the headset, such as an indicator light, may be used for prompting, for example, a preset color prompt or a flashing light may be used for prompting. Alternatively, a voice prompt may be used, for example, "You have not successfully worn the headset" may be broadcasted to indicate that the headset is worn abnormally.

[0066] Step 406: in response to the earphone being in the wearing state, obtaining a cross-correlation value of the left earphone and a cross-correlation value of the right earphone.

[0067] In a possible implementation, after the wearing state of the earphone is determined, the wearing consistency of the left and right ears can also be determined by the cross-correlation value between the audio signals. Therefore, after the wearing state of the earphone is determined, the first cross-correlation value, the second cross-correlation value, and the third cross-correlation value of the left earphone are obtained, and the first cross-correlation value, the second cross-correlation value, and the third cross-correlation value of the right earphone are obtained.

[0068] Step 407: In response to the similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being greater than a similarity threshold, it is determined that the earphone is in a normal wearing state.

[0069] After the mutual correlation values ​​of the left earphone and the right earphone are obtained, the similarity of the mutual correlation values ​​can be determined according to the mutual correlation values ​​corresponding to the left earphone and the right earphone. For example, the similarity between the first mutual correlation values ​​is determined according to the difference between the first mutual correlation value of the left earphone and the first mutual correlation value of the right earphone, and when the difference is less than a preset threshold, it can be determined that the similarity is greater than the similarity threshold.

[0070] Optionally, the wearing consistency of the left and right ears of the earphones can be determined according to the difference between the first mutual correlation values, the difference between the second mutual correlation values, and the difference between the third mutual correlation values. When the difference between the first mutual correlation values, the difference between the second mutual correlation values, and the difference between the third mutual correlation values ​​are all less than the difference threshold, it can be determined that the similarity of the mutual correlation values ​​between the left earphone and the right earphone is greater than the similarity threshold, and it is determined that the left and right ears of the earphones are worn consistently and are in a normal wearing state.

[0071] Step 408: In response to the similarity between the cross-correlation value of the left earphone and the cross-correlation value of the right earphone being less than a similarity threshold, it is determined that the earphone is in an abnormal wearing state.

[0072] In a possible implementation, when a difference between the first mutual correlation values, the second mutual correlation values, and the third mutual correlation values ​​of the left earphone and the right earphone is greater than a difference threshold, it is determined that the similarity of the mutual correlation values ​​is less than the similarity threshold, and it is determined that the left and right ears are worn inconsistently and are in an abnormal wearing state.

[0073] In this embodiment, by setting corresponding thresholds for the cross-correlation values ​​between different signals, the headphone status is determined based on the relationship between the first cross-correlation value and the first threshold, the relationship between the second cross-correlation value and the second threshold, and the relationship between the third cross-correlation value and the third threshold, thereby improving the accuracy of headphone wearing detection.

[0074] In this embodiment, after detecting that the earphone is in a wearing state, the third cross-correlation value can be further used to determine the tightness of the earphone wearing, and when the wearing is abnormal, the user can be prompted to make adjustments, thereby ensuring the sound quality and noise reduction effect of the earphone. The consistency of wearing of the left and right earphones can also be detected by the similarity of the cross-correlation values ​​between the left earphone and the right earphone.

[0075] In another possible implementation, to improve detection efficiency, when the first mutual correlation value meets certain conditions, it is possible to determine whether the earphone is in the wearing state only by the first mutual correlation value, without determining the magnitude of the second mutual correlation value and the third mutual correlation value. That is, the above step 402 can be replaced by the following two steps:

[0076] Step 1: determining a first cross-correlation value between the second audio signal and the third audio signal;

[0077] First, the first mutual correlation value needs to be determined, and whether the preset condition is met is determined according to the size of the first mutual correlation value.

[0078] Step 2: In response to the first mutual correlation value being less than a fourth threshold, determining a second mutual correlation value between the first audio signal and the second audio signal, and a third mutual correlation value between the first audio signal and the third audio signal.

[0079] The first cross-correlation value indicates the degree of correlation between the audio signals collected by the feedforward microphone and the feedback microphone. When the audio signals collected by the feedforward microphone and the feedback microphone are large, it indicates that the two audio signals are relatively similar. It can be determined that the earphones are not worn, and there is no need to make a judgment based on the second cross-correlation value and the third cross-correlation value.

[0080] In a possible implementation, when the first mutual correlation value is greater than the fourth threshold, it indicates that the second audio signal and the third audio signal are highly correlated, and it can be directly determined that the earphone is not worn. When the first mutual correlation value is less than the fourth threshold, the earphone state cannot be directly determined based on the first mutual correlation value, and therefore the second mutual correlation value and the third mutual correlation value need to be determined.

[0081] In a possible implementation, the audio signal may be converted from a time domain signal to a frequency domain signal, a cross-correlation value may be determined based on the frequency domain signal, and data processing may be performed on the cross-correlation value to ensure detection accuracy, which will be described below with an exemplary embodiment.

[0082] Please refer to Figure 5 , which shows a method flow chart of a headphone wearing detection method provided by an exemplary embodiment of the present application, and the method may include the following steps.

[0083] Step 501: Play a first audio signal through a speaker, and obtain a second audio signal collected by a feedforward microphone and a third audio signal collected by a feedback microphone.

[0084] The embodiment of this step may refer to the above step 201, which will not be described in detail in this embodiment.

[0085] Step 502: Perform frame windowing and short-time Fourier transform on the first audio signal, the second audio signal and the third audio signal to obtain a first spectrum function of the first audio signal, a second spectrum function of the second audio signal and a third spectrum function of the third audio signal.

[0086] Optionally, in an embodiment of the present application, the cross-correlation value between signals can be determined based on the frequency domain signal of the audio signal. First, the audio signal is converted into an analog-to-digital conversion, and the analog signal is converted into a digital signal. The conversion of the signal from the time domain to the frequency domain can be achieved through Fourier transform. Since the input signal of the Fourier transform requires a stable signal, the audio signal can be framed. The audio signal in one frame can be regarded as a stable signal, so Fourier transform can be performed on each frame of the audio signal. For example, the audio signal can be divided into signals with a length of 20ms per frame. In order to achieve smooth processing between frames, the audio signal can be windowed. Optionally, a rectangular window can be added to each frame of the audio signal for signal processing. Among them, the rectangular window function is as follows:

[0087]

[0088] Wherein, N is the window length, that is, the amount of data contained in each audio window.

[0089] After the frame processing, short-time Fourier transform can be performed to convert the time domain signal into a frequency domain signal.

[0090] Among them, the short-time Fourier transform method is as follows:

[0091]

[0092] Wherein, n refers to discrete time, ω=2πk / N, k=0, 1, ..., N-1, N is the Fourier transform length, that is, the number of frequency points of Fourier transform, and x(m) is the input signal of the mth frame.

[0093] The first spectrum function, the second spectrum function and the third spectrum function can be obtained by performing short-time Fourier transform on the first audio signal, the second audio signal and the third audio signal.

[0094] Step 503: determine the cross-correlation values ​​between any two of the first audio signal, the second audio signal, and the third audio signal based on the first spectrum function, the second spectrum function, and the third spectrum function.

[0095] In a possible implementation, the cross-correlation value between two signals may be determined by the first spectrum function, the second spectrum function, and the third spectrum function. Determining the cross-correlation value may include the following steps:

[0096] Step 1: for any two groups of audio signals, determine the frequency point cross-correlation values ​​of the spectrum functions corresponding to the two groups of audio signals at n frequency points.

[0097] Assume that the first spectrum function is X1(m, k), the second spectrum function is X2(m, k), and the third spectrum function is X3(m, k). Then the cross-correlation values ​​of the second audio signal and the third audio signal at n frequency points are calculated as follows:

[0098]

[0099] in, is the conjugate complex number of X3(m, k). Corr1(m, k) is the cross-correlation value between the second audio signal and the third audio signal at the kth frequency point in the mth frame signal.

[0100] The cross-correlation values ​​of the first audio signal and the second audio signal at n frequency points are calculated as follows:

[0101]

[0102] in, is the conjugate complex number of X2(m, k).

[0103] The cross-correlation values ​​of the first audio signal and the third audio signal at n frequency points are calculated as follows:

[0104]

[0105] in, is the conjugate complex number of X3(m, k).

[0106] After the cross-correlation values ​​between the two signals are obtained, the cross-correlation values ​​need to be normalized. Taking the cross-correlation values ​​of the first audio signal and the second audio signal at n frequency points as an example, the processing method is:

[0107]

[0108] After normalization, in order to ensure data smoothness and filter out disturbances, the normalized cross-correlation value can be smoothed as follows:

[0109] Corr_smooth(m,k)=α*Corr_smooth(m-1,k)+(1-α)*Corr_norm(m,k)

[0110] Wherein, α is a smoothing factor, and 0<α<1.

[0111] Step 2: Perform weighted processing on the cross-correlation values ​​of n frequency points to obtain the cross-correlation value between the two groups of audio signals, wherein the weight of the cross-correlation value of the frequency points is related to the commonness of the frequency points.

[0112] In a possible implementation, after determining the frequency point cross-correlation values ​​of two groups of audio signals at n frequency points, the headphone state of the headphone can be determined according to the frequency point cross-correlation values ​​at each frequency point and the corresponding thresholds. That is, the first frequency point cross-correlation values ​​of the second audio signal and the third audio signal at the n frequency points are respectively compared with the first threshold, the second frequency point cross-correlation values ​​of the first audio signal and the second audio signal at the n frequency points are respectively compared with the second threshold, and the third frequency point cross-correlation values ​​of the first audio signal and the third audio signal at the n frequency points are respectively compared with the third threshold. When the first frequency point cross-correlation values ​​of k frequency points among the n frequency points are less than the first threshold, the second frequency point cross-correlation values ​​of the k frequency points are less than the second threshold, and the third frequency point cross-correlation values ​​of the k frequency points are greater than the third threshold, it can be determined that the headphone is in the wearing state. When there is no k frequency point whose first frequency point cross-correlation value is less than the first threshold, the second frequency point cross-correlation value is less than the second threshold, or the third frequency point cross-correlation value is greater than the third threshold, it is determined that the headphone is in the not-worn state.

[0113] However, the process of comparing the frequency point cross-correlation values ​​at n frequency points with the threshold is cumbersome and the judgment efficiency is low. Therefore, in another possible implementation, the frequency point cross-correlation values ​​at n frequency points are weighted to obtain the cross-correlation values ​​between the two groups of audio signals, and the weighted cross-correlation values ​​are compared with the respective thresholds to judge the headphone status of the headphone.

[0114] Optionally, the weight of the frequency point cross-correlation value is related to the frequency point commonness. The sound in the low frequency band is more common, while the sound in the high frequency band is uncommon. Therefore, when the frequency point is in the low frequency band, the weight can be relatively large, and when the frequency point is in the high frequency band, the weight can be relatively small. When performing weighted processing, the corresponding weight of the frequency point can be determined according to the corresponding relationship between the frequency band where the frequency point is located and the weight. Schematically, the corresponding relationship between the frequency band and the weight can be shown in Table 1:

[0115] Table 1

[0116] Frequency band (Hz) Weight 0-1000 60％ 1000-3000 30％ 3000-20000 10％

[0117] Step 504: determine the earphone status based on the cross-correlation value.

[0118] The implementation of this step can refer to the above step 403, which will not be described in detail in this embodiment.

[0119] In this embodiment, the spectral function of the audio signal is obtained by framing, windowing and Fourier transforming the audio signal, the cross-correlation value is determined according to the spectral function, and the cross-correlation value is normalized and smoothed to ensure data accuracy, thereby ensuring the accuracy of headphone wearing detection.

[0120] In combination with the above embodiments, in an illustrative example, the headphone wearing detection process is as follows: Figure 6 The process may include the following steps:

[0121] Step 1, playing a first audio signal through a speaker, and collecting a second audio signal through a feedforward microphone and a third audio signal through a feedback microphone;

[0122] Step 2, performing frame division and windowing processing on the audio signal respectively;

[0123] Step 3, performing short-time Fourier transform on the audio signal after the frame-by-frame windowing process;

[0124] Step 4, determining a first mutual correlation value based on the second audio signal after short-time Fourier transform and the third audio signal, determining a second mutual correlation value based on the first audio signal after short-time Fourier transform and the second audio signal, and determining a third mutual correlation value based on the first audio signal after short-time Fourier transform and the third audio signal;

[0125] Step 5, comparing the first mutual correlation value with the first threshold, comparing the second mutual correlation value with the second threshold, and comparing the third mutual correlation value with the third threshold;

[0126] Step 6: When the first mutual correlation value is less than the first threshold, the second mutual correlation value is less than the second threshold, and the third mutual correlation value is greater than the third threshold, it is determined that the earphone is in the wearing state.

[0127] Please refer to Figure 7 , which shows a structural block diagram of an earphone wearing detection device provided by an embodiment of the present application. The device can be implemented as all or part of the earphone through software, hardware or a combination of both. The device includes:

[0128] A first acquisition module 701 is used to play a first audio signal through the speaker, and acquire a second audio signal collected by the feedforward microphone and a third audio signal collected by the feedback microphone;

[0129] A first determining module 702, configured to determine a cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal, wherein the cross-correlation value is used to indicate a correlation degree between the audio signals;

[0130] The second determining module 703 is configured to determine a headphone state of the headphone based on the cross-correlation value, where the headphone state includes a wearing state and a not-wearing state.

[0131] Optionally, the first determining module 702 is further configured to:

[0132] determining a first cross-correlation value between the second audio signal and the third audio signal, a second cross-correlation value between the first audio signal and the second audio signal, and a third cross-correlation value between the first audio signal and the third audio signal;

[0133] Optionally, the second determining module 703 is further configured to:

[0134] The earphone state is determined based on the first mutual correlation value, the second mutual correlation value, the third mutual correlation value, and thresholds corresponding to the respective mutual correlation values.

[0135] Optionally, the second determining module 703 includes:

[0136] a first determining unit, configured to determine that the headset is in a wearing state in response to the first mutual correlation value being less than a first threshold, the second mutual correlation value being less than a second threshold, and the third mutual correlation value being greater than a third threshold;

[0137] The second determining unit is configured to determine that the headset is in an unworn state in response to the first mutual correlation value being greater than a first threshold, the second mutual correlation value being greater than a second threshold, or the third mutual correlation value being less than a third threshold.

[0138] Optionally, the first determining module 702 includes:

[0139] a third determining unit, configured to determine the first mutual correlation value between the second audio signal and the third audio signal;

[0140] The fourth determining unit is configured to determine the second mutual correlation value between the first audio signal and the second audio signal and the third mutual correlation value between the first audio signal and the third audio signal in response to the first mutual correlation value being less than a fourth threshold.

[0141] Optionally, the device further comprises:

[0142] a third determining module, configured to determine that the headset is in a normal wearing state in response to the headset being in a wearing state and the third mutual correlation value being greater than a fifth threshold;

[0143] The fourth determination module is used to determine that the headset is in an abnormal wearing state and to give a prompt in response to the headset being in a wearing state and the third mutual correlation value being less than a fifth threshold.

[0144] Optionally, the device further comprises:

[0145] a processing module, configured to perform frame windowing processing and short-time Fourier transform on the first audio signal, the second audio signal and the third audio signal, respectively, to obtain a first spectrum function of the first audio signal, a second spectrum function of the second audio signal and a third spectrum function of the third audio signal;

[0146] The first determining module 702 is further configured to:

[0147] The cross-correlation values ​​between any two of the first audio signal, the second audio signal, and the third audio signal are determined based on the first spectral function, the second spectral function, and the third spectral function.

[0148] The first determining module 702 further includes:

[0149] A fifth determining unit, configured to determine, for any two groups of audio signals, frequency point cross-correlation values ​​of frequency spectrum functions corresponding to the two groups of audio signals at n frequency points;

[0150] A processing unit is used to perform weighted processing on the n frequency point mutual correlation values ​​to obtain the mutual correlation value between the two groups of audio signals, wherein the weight of the frequency point mutual correlation value is related to the commonness of the frequency point.

[0151] Optionally, the device further comprises:

[0152] A second acquisition module, configured to acquire a cross-correlation value of a left earphone and a cross-correlation value of a right earphone in response to the earphone being in a wearing state;

[0153] a fifth determining module, configured to determine that the earphone is in a normal wearing state in response to a similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being greater than a similarity threshold;

[0154] The sixth determination module is configured to determine that the earphone is in an abnormal wearing state in response to the similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being less than a similarity threshold.

[0155] To summarize, in the embodiment of the present application, an audio signal is played through a speaker, and audio signals are collected by using a feedforward microphone and a feedback microphone arranged at different positions of the earphone, so that the correlation degree between each two of the three audio signals is obtained according to the cross-correlation values ​​between each two of the three audio signals, and then whether the earphone is in a wearing state is judged according to the correlation degree between the signals, thereby improving the accuracy of earphone wearing detection, and the correlation between the detection signals is not affected by environmental factors or false touch of the sensor, and the detection function is stable, which helps to reduce the probability of misjudgment.

[0156] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the headphone wearing detection method as described in the above embodiment.

[0157] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the headphone wearing detection method provided in the above embodiment.

[0158] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0159] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting earphone wearing, characterized in that: The method is used for a headset provided with a feedforward microphone, a feedback microphone and a speaker, and the method comprises: Playing a first audio signal through the speaker, and acquiring a second audio signal collected by the feedforward microphone and a third audio signal collected by the feedback microphone; Determine a cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal, wherein the cross-correlation value is used to indicate a correlation degree between the audio signals; Determine a headphone state of the headphone based on the cross-correlation value, where the headphone state includes a wearing state and a not-wearing state; In response to the earphone being in a wearing state, acquiring the cross-correlation value of the left earphone and the cross-correlation value of the right earphone; In response to a similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being greater than a similarity threshold, determining that the earphone is in a normal wearing state; In response to the similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being less than the similarity threshold, it is determined that the earphone is in an abnormal wearing state.

2. The method according to claim 1, characterized in that: The determining of the cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal comprises: determining a first cross-correlation value between the second audio signal and the third audio signal, a second cross-correlation value between the first audio signal and the second audio signal, and a third cross-correlation value between the first audio signal and the third audio signal; The determining the headphone state of the headphone based on the cross-correlation value comprises: The earphone state is determined based on the first mutual correlation value, the second mutual correlation value, the third mutual correlation value, and thresholds corresponding to the respective mutual correlation values.

3. The method according to claim 2, characterized in that The determining the earphone state based on the first mutual correlation value, the second mutual correlation value, the third mutual correlation value, and thresholds corresponding to the mutual correlation values ​​respectively includes: In response to the first mutual correlation value being less than a first threshold, the second mutual correlation value being less than a second threshold, and the third mutual correlation value being greater than a third threshold, determining that the headset is in a wearing state; In response to the first mutual correlation value being greater than a first threshold, the second mutual correlation value being greater than a second threshold, or the third mutual correlation value being less than a third threshold, it is determined that the headset is in an unworn state.

4. The method according to claim 2, characterized in that: The determining a first mutual correlation value between the second audio signal and the third audio signal, a second mutual correlation value between the first audio signal and the second audio signal, and a third mutual correlation value between the first audio signal and the third audio signal comprises: determining the first cross-correlation value between the second audio signal and the third audio signal; In response to the first mutual correlation value being less than a fourth threshold, the second mutual correlation value between the first audio signal and the second audio signal and the third mutual correlation value between the first audio signal and the third audio signal are determined.

5. The method according to any one of claims 2 to 4, characterized in that: After determining the headphone state of the headphone based on the cross-correlation value, the method further includes: In response to the earphone being in a wearing state and the third mutual correlation value being greater than a fifth threshold, determining that the earphone is in a normal wearing state; In response to the earphone being in a wearing state and the third mutual correlation value being less than a fifth threshold, it is determined that the earphone is in an abnormal wearing state, and a prompt is given.

6. The method according to any one of claims 1 to 4, characterized in that: Before determining the cross-correlation values ​​between any two of the first audio signal, the second audio signal, and the third audio signal, the method further includes: Performing frame windowing processing and short-time Fourier transform on the first audio signal, the second audio signal, and the third audio signal respectively to obtain a first spectrum function of the first audio signal, a second spectrum function of the second audio signal, and a third spectrum function of the third audio signal; The determining of the cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal comprises: The cross-correlation values ​​between any two of the first audio signal, the second audio signal, and the third audio signal are determined based on the first spectral function, the second spectral function, and the third spectral function.

7. The method according to claim 6, characterized in that The determining, based on the first spectrum function, the second spectrum function and the third spectrum function, the cross-correlation value between any two of the first audio signal, the second audio signal and the third audio signal comprises: For any two groups of audio signals, determine the frequency point cross-correlation values ​​of the frequency spectrum functions corresponding to the two groups of audio signals at n frequency points; The n frequency point mutual correlation values ​​are weighted to obtain the mutual correlation value between the two groups of audio signals, wherein the weight of the frequency point mutual correlation value is related to the commonness of the frequency point.

8. An earphone wearing detection device, characterized in that: The device is used for a headset provided with a feedforward microphone, a feedback microphone and a loudspeaker, and the device comprises: A first acquisition module, configured to play a first audio signal through the speaker, and acquire a second audio signal collected by the feedforward microphone, and a third audio signal collected by the feedback microphone; A first determining module, configured to determine a cross-correlation value between any two of the first audio signal, the second audio signal, and the third audio signal, wherein the cross-correlation value is used to indicate a correlation degree between the audio signals; A second determining module, configured to determine a headphone state of the headphone based on the cross-correlation value, wherein the headphone state includes a wearing state and a not-wearing state; A second acquisition module, configured to acquire the cross-correlation value of the left earphone and the cross-correlation value of the right earphone in response to the earphone being in a wearing state; a fifth determining module, configured to determine that the earphone is in a normal wearing state in response to a similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being greater than a similarity threshold; The sixth determination module is configured to determine that the earphone is in an abnormal wearing state in response to the similarity between the mutual correlation value of the left earphone and the mutual correlation value of the right earphone being less than the similarity threshold.

9. A headset, characterized in that: The headset includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the headset wearing detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the headphone wearing detection method according to any one of claims 1 to 7.

Images