Headphone control method, device, electronic device and storage medium

Abstract

The present disclosure relates to a method for controlling an earphone, wherein the earphone includes a feedback microphone, and the method includes: collecting an audio signal in an ear canal through the feedback microphone to obtain an ear canal audio signal; the ear canal audio signal includes: when the earphone is worn by a user and the user performs an interactive operation in a target area of ​​his or her head, the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction and an audio signal is generated; feature extraction is performed on the ear canal audio signal to obtain audio signal feature parameters; the tightness level of the earphone in the current wearing state is obtained; based on the audio signal feature parameters and the tightness level, an audio feature to be identified is generated; the audio feature to be identified is input into a preset interactive operation recognition model, and a recognition result is output, wherein the recognition result includes an interactive operation identifier; a control instruction corresponding to the interactive operation identifier is determined; and the playback state of the speaker is controlled according to the control instruction.

Description

Technical Field

[0001] The present disclosure relates to the field of control technology, and in particular to an earphone control method, device, electronic device and storage medium. Background Art

[0002] Headphones have become an important audio player in daily life. Music and various audio can be played through headphones. When various audio or video playback devices are inconvenient or cannot be played externally, headphones can be connected to audio / video playback devices to play the corresponding audio through the headphones.

[0003] There are many types of headphones, with different functions and operation methods. Different interactive operation methods will bring different user experiences. Summary of the invention

[0004] The present disclosure provides an earphone control method, an apparatus, an electronic device and a storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided a method for controlling an earphone, wherein the earphone comprises a feedback microphone, and the method comprises: collecting an audio signal in the ear canal through the feedback microphone to obtain an ear canal audio signal; wherein the ear canal audio signal comprises: when the earphone is worn by a user and the user performs an interactive operation in a target area of ​​his or her head, an audio signal is generated when the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction; performing feature extraction on the ear canal audio signal to obtain audio signal feature parameters; obtaining the tightness level of the earphone in the current wearing state; generating an audio feature to be identified based on the audio signal feature parameters and the tightness level; inputting the audio feature to be identified into a preset interactive operation recognition model and outputting a recognition result, wherein the recognition result includes an interactive operation identifier; determining a control instruction corresponding to the interactive operation identifier; and controlling the playback state of the earphone according to the control instruction.

[0006] In one embodiment, the tightness of the earphone in the current wearing state is detected at predetermined intervals to obtain the tightness level, and the tightness level is stored.

[0007] In one embodiment, the earphone includes a speaker, and the detection of the tightness of the earphone in a current wearing state to obtain the tightness level includes: collecting an ultrasonic reflection signal in the ear canal through the feedback microphone, wherein the ultrasonic reflection signal is an audio signal obtained after the ultrasonic wave is reflected by the ear canal when propagating in the ear canal, and the ultrasonic wave is emitted by the speaker at a predetermined frequency; calculating the energy of the ultrasonic reflection signal to obtain ultrasonic reflection energy; and obtaining the tightness level according to the ultrasonic reflection energy and pre-stored ultrasonic emission energy.

[0008] In one embodiment, calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy includes: obtaining the ultrasonic reflection energy according to the energy of each sampling point in at least one signal frame of the ultrasonic reflection signal.

[0009] In one embodiment, obtaining the tightness level according to the ultrasonic reflection energy and the pre-stored ultrasonic transmission energy includes: determining the tightness level according to the energy difference between the ultrasonic reflection energy and the ultrasonic transmission energy; wherein there is a preset mapping relationship between the energy difference and the tightness level.

[0010] In one embodiment, before calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy, the method further includes: filtering the ultrasonic reflection signal to obtain the filtered ultrasonic reflection signal.

[0011] In one embodiment, before extracting features from the ear canal audio signal to obtain audio signal feature parameters, the method further includes: performing low-pass filtering on the ear canal audio signal to obtain the filtered ear canal audio signal.

[0012] In one embodiment, the interactive operation recognition model is obtained by pre-training an initial neural network model using a training sample set in a machine learning manner.

[0013] In one embodiment, the training sample set includes a positive sample set, and the positive sample set includes multiple positive samples; each of the positive samples includes: a target interaction operation audio feature, the tightness level and a first label; the target interaction operation audio feature is an audio feature obtained by collecting a sound signal in the ear canal through the earphone and processing the sound signal when the user performs a target operation in a target area while wearing earphones; the first label is used to identify the target operation and the tightness level.

[0014] In one embodiment, the training sample set also includes a negative sample set, which includes multiple negative samples; each of the negative samples includes: a non-target interactive operation audio feature, the tightness level, and a second label; the non-target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphones and processing the sound signals when the user performs non-target operations in the target area while wearing earphones; the second label is used to identify the non-target operation and the tightness level.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided an earphone control device, wherein the earphone includes a feedback microphone, and the device includes: an ear canal audio signal acquisition module, configured to acquire an audio signal in the ear canal through the feedback microphone to obtain an ear canal audio signal; wherein the ear canal audio signal includes: when the earphone is worn by a user and the user performs an interactive operation in a target area of ​​his / her head, the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction and the audio signal is generated; an audio signal characteristic parameter determination module, configured to extract features of the ear canal audio signal to obtain audio signal characteristic parameters; a tightness level determination module, configured to obtain the tightness level of the earphone in a current wearing state; a to-be-identified audio feature determination module, configured to generate an to-be-identified audio feature according to the audio signal characteristic parameter and the tightness level; a recognition result determination module, configured to input the to-be-identified audio feature into a preset interactive operation recognition model and output a recognition result, wherein the recognition result includes an interactive operation identifier; a control instruction determination module, configured to determine a control instruction corresponding to the interactive operation identifier; and a control module, configured to control the playback state of the earphone according to the control instruction.

[0016] A third aspect of the present disclosure provides an electronic device, including:

[0017] A processor and a memory for storing executable instructions that can be run on the processor, wherein: when the processor is used to run the executable instructions, the executable instructions execute the method described in any one of the above embodiments.

[0018] According to a fourth aspect of the embodiments of the present disclosure, a non-temporary computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0019] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0020] The disclosed embodiments can be applied to headphones, which have a feedback microphone, and the audio signal in the ear canal is collected by the feedback microphone to obtain an ear canal audio signal, and the ear canal audio signal includes an audio signal generated by the vibration generated by the interactive operation being transmitted to the ear canal through bone conduction when the headset is worn by the user and the user performs an interactive operation in the target area of ​​his or her head. Then, feature extraction is performed on the ear canal audio signal to obtain audio signal feature parameters. The tightness level of the headset in the current wearing state is obtained, and the audio features to be identified obtained based on the audio signal feature parameters and the tightness level are input into a preset interactive operation recognition model to obtain a recognition result, which includes an interactive operation identifier. Then, the playback state of the headset is controlled according to the control instruction determined to correspond to the interactive operation identifier.

[0021] The solution of this embodiment does not need to install a pressure sensor or other operation sensor in the earphone to detect the user's operation, which saves the internal space of the earphone, and also saves the cost of the earphone, reduces the size of the earphone, and provides installation space for other components. On the other hand, it reduces the problem of difficulty in operation caused by the user's inability to accurately touch the sensor in the earphone that detects the user's operation, reduces the user's operation difficulty, and the user can control the earphone by operating the target area in the limbs, such as acting on the head, face, or the earphone in the wearing state, etc., which provides a larger operation space, thereby improving the user's experience.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0024] Figure 1 is a flowchart of a headset control method according to an exemplary embodiment;

[0025] Figure 2 is a schematic diagram of an earphone according to an exemplary embodiment;

[0026] Figure 3 is a schematic diagram showing a headset in a wearing state according to an exemplary embodiment;

[0027] Figure 4 is a schematic diagram showing amplification of an ear canal audio signal in an ear canal by an occlusion effect according to an exemplary embodiment;

[0028] Figure 5is a schematic diagram showing a method of determining a tightness level according to an exemplary embodiment;

[0029] Figure 6 is a schematic structural diagram of an earphone control device according to an exemplary embodiment;

[0030] Figure 7 The present invention is a block diagram of a terminal device according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices consistent with some aspects of the present disclosure as detailed in the appended claims.

[0032] With the development of technology, headphones provide more and more functions, such as true wireless stereo (TWS) headphones, which have multiple interactive functions. For example, when users wear Bluetooth headphones, they need multiple ways of interaction, such as sliding up and down the headphone handle to adjust the volume. Since the headphone cavity is very small and users have many interactive needs, more dimensional interaction methods are needed to provide more natural interactive experiences.

[0033] Usually, the interactive function of headphones is usually realized by sensors. A pressure sensor is set on the headphone handle. Operations such as clicking or sliding on the sensor will produce different types of pressure. By analyzing the pressure, the headphones can identify the user's operation behavior and respond with corresponding functions.

[0034] Because users cannot see the earphones worn on their ears, their fingers cannot accurately position the sensors, which makes operation difficult. In addition, the newly added sensors require new hardware costs.

[0035] refer to Figure 1 , is a flow chart of a headphone control method provided by the technical solution of the present disclosure. The headphone includes a feedback microphone, and the method includes:

[0036] Step S100, collecting the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal; wherein the ear canal audio signal includes: when the earphone is worn by the user, when the user performs an interactive operation in the target area of ​​his or her head, the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction, and the audio signal is generated.

[0037] Step S200: extract features of the ear canal audio signal to obtain audio signal feature parameters.

[0038] Step S300, obtaining the tightness level of the earphone in the current wearing state.

[0039] Step S400: generating an audio feature to be identified according to the audio signal feature parameter and the tightness level.

[0040] Step S500: input the audio feature to be recognized into a preset interactive operation recognition model, and output a recognition result, wherein the recognition result includes an interactive operation identifier.

[0041] Step S600: determining a control instruction corresponding to the interactive operation identifier.

[0042] Step S700: Control the playing state of the earphone according to the control instruction.

[0043] The earphones can be of different shapes, including in-ear, semi-in-ear, and head-mounted earphones.

[0044] The feedback microphone in the earphone can be located near the sound output channel of the earphone. When the earphone is worn, the feedback microphone is located in the ear canal and can collect audio signals in the ear canal, such as in-ear earphones. When the earphone is other types of earphones, such as semi-in-ear earphones and headphones, the feedback microphone can collect audio signals in the ear canal when the earphone is worn.

[0045] For step S100, when the earphone is in the wearing state, the ear canal is blocked to a certain extent, forming a certain degree of ear blocking effect. The reason for this is that part of the sound is conducted to the inner ear through the human bones. For example, when the user performs interactive operations in the target area of ​​his head, the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction to generate an audio signal. When the earphone is not in the wearing state, part of the sound from bone conduction also diffuses outward through the outer ear, but when the earphone is in the wearing state, the ear canal through which the sound diffuses outward is blocked to a certain extent, reducing the amount of sound from bone conduction that diffuses outward through the ear canal, forming a certain degree of occlusion effect. The sound characteristics generated by the occlusion effect are characterized by low-frequency enhancement and high-frequency attenuation.

[0046] Since the earphones block the ear canal to a certain extent, forming different degrees of ear-blocking effect, after the ear-blocking effect occurs, the earphones block external audio signals from entering the ear canal, reducing the impact of external audio signals on the audio signals in the ear canal. The feedback microphone can collect the audio signal in the ear canal to obtain the ear canal audio signal, which includes the audio signal generated when the earphone is in a state of being worn by the user and the user performs interactive operations in the target area of ​​his or her head, and the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction. The target area can be the user's head, face, neck and other areas. The interactive operations here can include operations such as touch and tap, which can be a single touch, a single tap, or continuous touch and connected tapping. The interactive operation can also be an operation acting on the earphones in a worn state, and the vibration generated by the interactive operation can also be transmitted to the ear canal through bone conduction to generate an audio signal.

[0047] After a certain degree of ear-blocking effect is formed, the vibration generated by the interactive operation can be conducted to the ear canal through the bones of the face, head, etc., to generate an audio signal in the first frequency range, such as an audio signal of 100 Hz to 1000 Hz. The ear-blocking effect can amplify the audio signal in the first frequency range, thereby facilitating the feedback microphone to collect the ear canal audio signal. The feedback microphone can directly collect the audio signal in the first frequency range as the ear canal audio signal, or collect the amplified audio signal of the first frequency as the ear canal audio signal.

[0048] For step S200, after obtaining the ear canal audio signal, feature extraction can be performed on the ear canal audio signal to obtain audio signal feature parameters. There are many ways to extract features, such as extracting corresponding features through a feature extraction algorithm, and the obtained audio signal feature parameters can be Mel spectrum coefficients and Mel cepstral coefficients (MFCC), etc. Mel spectrum coefficients and Mel cepstral coefficients can both be 40-dimensional feature parameters. Of course, other features of the ear canal audio signal can also be used.

[0049] For step S300, when the earphone is in the wearing state, the tightness level of the earphone in the current wearing state can be determined by the feedback microphone, and the specific determination process is not limited here. For example, the feedback microphone can collect the audio signal emitted by the speaker of the earphone, and the tightness level of the earphone in the current wearing state can be determined based on the audio signal emitted by the speaker collected by the feedback microphone. Different wearing states may correspond to different tightness levels.

[0050] In one embodiment, the gap between the earphone and the ear canal may be different corresponding to different tightness levels. A larger gap indicates a smaller tightness level, and a smaller gap indicates a larger tightness level. A larger tightness level indicates that the earphone is worn tighter, and a smaller tightness level indicates that the earphone is worn looser. Of course, a smaller tightness level may indicate that the earphone is worn tighter, and a larger tightness level may indicate that the earphone is worn looser. The relationship between the tightness level and the tightness of the earphone can be determined according to actual usage requirements.

[0051] In one embodiment, the tightness level can be divided into levels according to actual usage requirements, such as level one, level two...level five, etc. The higher the level, the greater the tightness level. The higher the level, the smaller the tightness level. It can be determined according to actual needs.

[0052] Of course, there may also be other methods for determining the tightness level of the headset in the current wearing state based on the feedback microphone. This embodiment only cites one of them. As long as the tightness level is determined by the feedback microphone, it is within the protection scope described above.

[0053] In one embodiment, there is no necessary order relationship between step S300 and step S100, and there is no necessary order relationship between step S300 and step S200. Step S100 and step S200 may be performed first, and then step S300, or step S300 may be performed first, and then step S100 and step S200. Step S300 may be in parallel with step S100 and step S200.

[0054] For step S400, after the tension level and the audio signal characteristic parameters are determined, an audio feature to be identified can be generated according to the tension level and the audio signal parameters. For example, the tension level and the audio signal parameters can be used together as an audio feature to be identified, and the audio feature to be identified includes information of two dimensions, the tension level and the audio signal parameters, for determining the interactive operation identifier corresponding to the interactive operation.

[0055] Different audio features to be identified are obtained according to different tension levels and audio signal characteristic parameters. When at least one of the tension level and the audio signal characteristic parameter changes, the audio features to be identified change, which can reduce the situation where the audio features to be identified do not change when the tension level and the audio signal characteristic parameter change at the same time, thereby improving the accuracy of the recognition result.

[0056] For step S500, after the audio feature to be identified is generated, an interactive operation identifier can be determined according to the audio feature to be identified. There is a corresponding relationship between the interactive operation identifier and the interactive operation. After the interactive operation identifier is determined, the corresponding interactive operation can be determined according to the interactive operation identifier. The interactive operation identifier can represent the corresponding interactive operation, for example, one interactive operation identifier corresponds to one interactive operation, and there is a one-to-one correspondence between the interactive operation identifier and the interactive operation.

[0057] For example, the interactive operation recognition model can be used to input the audio features to be recognized into the interactive operation recognition model, and the interactive operation recognition model outputs a recognition result, which includes an interactive operation identifier, and the interactive operation identifier corresponds to an interactive operation. The interactive operation recognition model is a model trained in advance according to the interactive operation identifier and the audio features to be recognized.

[0058] Of course, it can also be determined by other means, such as a mapping table, which includes a mapping relationship between the audio features to be identified and the interactive operation identifier. By searching the mapping table, the interactive operation identifier corresponding to the interactive operation acting on the target area can be determined based on the audio features to be identified.

[0059] For example, the tightness is level 2, and the audio signal feature parameter is M. The audio features to be identified obtained based on the tightness level 2 and the audio signal feature parameter M are input into the interactive operation model, or a mapping table is searched, and the recognition result of the interactive operation identifier of clicking the face can be determined.

[0060] Different interactive operation identifiers acting on the target area are determined according to different tightness levels and different target audio signals.

[0061] For step S600, after the interactive operation identifier is determined, the corresponding control instruction can be determined according to the interactive operation identifier, and then the playback state of the headset can be controlled according to the control instruction. There is a preset correspondence between the interactive operation identifier and the control instruction. After the interactive operation identifier is determined, the control instruction corresponding to the interactive operation identifier can be determined according to the preset correspondence between the interactive operation identifier and the control instruction.

[0062] Different interactive operations correspond to different interactive operation identifiers, and different interactive operation identifiers correspond to different control instructions. Different control instructions can control the headset differently.

[0063] For example, the interactive operation identifier X1 corresponds to the control instruction Y1, and the interactive operation identifier X1 is the identifier of the operation of single-clicking the face area. The interactive operation identifier X2 corresponds to the control instruction Y2, and the interactive operation identifier X2 is the identifier of the operation of double-clicking the face. The interactive operation identifier X3 corresponds to the control instruction Y3, and the interactive operation identifier X3 is the identifier of the operation of touching the face.

[0064] In one embodiment, there is a preset corresponding relationship between the interactive operation identifier and the control instruction, which may exist in the form of a relationship table, and the control instruction is determined by looking up the table.

[0065] For step S700, after the control instruction is determined, the playing state of the earphone can be controlled according to the control instruction. For example, if the control instruction is control instruction Y1 and the corresponding control operation is to increase the volume, the volume of the earphone can be increased according to control instruction Y1. If the control instruction is control instruction Y2 and the corresponding control operation is to decrease the volume, the volume of the earphone can be decreased according to control instruction Y1. If the control instruction is control instruction Y3 and the corresponding control operation is a pause operation, the playing of the earphone can be paused according to control instruction Y1.

[0066] This solution collects the ear canal audio signal of the ear canal through the feedback microphone, performs feature extraction and other analysis on the ear canal audio signal, identifies the user's interactive operation, and thus controls the earphone to respond to the corresponding function. By using the existing built-in microphone, there is no need for new hardware costs, and there is no need to install pressure sensors or other operation sensors in the earphone to detect the user's operation, which saves the internal space of the earphone, while also saving the cost of the earphone, reducing the size of the earphone, and providing installation space for other components. On the other hand, it reduces the problem of difficulty in operation caused by the user's inability to accurately touch the sensor in the earphone that detects the user's operation, reduces the user's operation difficulty, and reduces the probability of misoperation and invalid operation. The user can control the earphone by operating the target area in the limbs, such as acting on the head, face, or the earphone in the wearing state, which provides a larger operating space and more interactive methods, thereby improving the user's experience.

[0067] refer to Figure 2 , is a schematic diagram of an earphone, including a feedback microphone A. When the earphone is in a worn state, the feedback microphone A can be located in the ear canal. The earphone may also include a feedforward microphone B. The feedforward microphone B can be located on the earphone handle. When the earphone is in a worn state, the feedforward microphone is located outside the ear canal and can collect ambient audio signals from the external environment. The earphone may also include a call microphone C to collect audio signals emitted by the user in a call state. The audio signal collected by the feedback microphone A has a higher signal-to-noise ratio than that collected by the feedforward microphone B, so the feedback microphone A collects ear canal audio signals with less noise and higher quality.

[0068] When the earphones are head-mounted earphones, a certain degree of ear-blocking effect will also be formed. When the earphones are worn, the feedback microphone A can be located outside the ear canal or facing the ear canal to collect the ear canal audio signal.

[0069] refer to Figure 3, is a schematic diagram of a headset in a wearing state, where the headset 1 blocks the ear canal 2 to a certain extent, forming an ear blocking effect. The ear canal audio signal collected by the feedback microphone includes: the vibration generated by the interactive operation such as touch or tapping on the user's face or head area can be transmitted to the ear canal through bone conduction to generate an audio signal, that is, audio signal 3.

[0070] Figure 4 The figure is a schematic diagram of a blockage effect amplifying the ear canal audio signal in the ear canal. L1 is an energy curve diagram of an audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction without the blockage effect. L3 is an energy curve diagram of L1 under the blockage effect. L2 is another energy curve diagram of an audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction without the blockage effect. L4 is an energy curve diagram of L2 under the blockage effect. Obviously, in the range of about 100Hz to 1000Hz, the blockage effect will significantly amplify the energy of the same audio signal, while the frequency of the audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction is usually within this frequency range.

[0071] In one embodiment, after a certain degree of ear blocking effect is formed, the ear canal may also include noise signals in addition to the audio signals generated by the vibration generated by the interactive operation being transmitted to the ear canal through bone conduction when the earphone is worn by the user and the user performs interactive operations on the target area of ​​his / her head. That is, the ear canal audio signal may also include audio signals in addition to the audio signals generated by the vibration generated by the interactive operation being transmitted to the ear canal through bone conduction.

[0072] After the feedback microphone collects the ear canal audio signal, the ear canal audio signal can be filtered to obtain the filtered ear canal audio signal except for the audio signal generated by the vibration generated by the interactive operation and transmitted to the ear canal through bone conduction. Then, the filtered ear canal audio signal is feature extracted to obtain the audio signal feature parameters.

[0073] By performing the above-mentioned filtering operation on the ear canal audio signal, the influence of audio signals other than the audio signals generated by the vibration generated by the interactive operation being transmitted to the ear canal through bone conduction can be reduced, thereby improving the signal-to-noise ratio of the ear canal audio signal and the accuracy of the recognition result, which is conducive to more accurate determination of control instructions based on the recognition result, thereby improving the accuracy of controlling the headphones.

[0074] In another embodiment, when obtaining the tightness level of the earphone in the current wearing state, the tightness level of the earphone can be obtained at a predetermined time interval, and the tightness of the earphone in the current wearing state is detected at each predetermined time interval to obtain the tightness level of the earphone. After obtaining the tightness level of the earphone, the currently obtained tightness level of the earphone can also be stored.

[0075] By detecting the tightness of the earphone in the current wearing state once every predetermined time, the tightness level of the earphone in the current wearing state can be updated, and the identification result can be determined more accurately according to the updated tightness level.

[0076] In another embodiment, the earphone further has a speaker, which is used to output audio signals. For example, the speaker can be located near the earphone sound output channel. When the earphone is worn, the audio signal emitted by the speaker can be transmitted to the ear canal. Figure 2 , shows an earphone including a speaker D.

[0077] refer to Figure 5 , is a schematic diagram of determining the tightness level, step S300, obtaining the tightness level of the earphone in the current wearing state, including:

[0078] Step S301, collecting ultrasonic reflection signals in the ear canal through a feedback microphone, wherein the ultrasonic reflection signal is an audio signal obtained after ultrasonic waves are reflected by the ear canal when propagating in the ear canal, and the ultrasonic waves are emitted by a speaker at a predetermined frequency.

[0079] Step S302, calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy.

[0080] Step S303, obtaining the tightness level according to the ultrasonic reflection energy and the pre-stored ultrasonic transmission energy.

[0081] When determining the tightness level of the earphone in the current wearing state, ultrasonic waves are output through the speaker, and the ultrasonic waves propagate in the ear canal, and the ear canal can reflect the ultrasonic waves. The feedback microphone can collect the audio signal obtained after the ultrasonic waves propagate in the ear canal and are reflected by the ear canal, that is, the ultrasonic reflection signal. In this embodiment, the ultrasonic reflection signal collected by the feedback microphone is different from the ultrasonic signal output by the speaker.

[0082] In one embodiment, the speaker may emit ultrasonic waves at a predetermined frequency.

[0083] Since the ultrasonic signal and the ultrasonic reflection signal are different, and when the earphone is at different tightness levels, the ultrasonic reflection signal is also different, the tightness level of the earphone can be determined according to the difference between the ultrasonic signal and the ultrasonic reflection signal.

[0084] For example, the greater the tightness level, the smaller the gap between the earphone and the ear canal may be, the smaller the amount of ultrasonic waves transmitted to the outside of the ear canal, and the smaller the difference between the ultrasonic wave and the ultrasonic wave reflected signal. The smaller the tightness level, the larger the gap between the earphone and the ear canal, the greater the amount of ultrasonic waves transmitted to the outside of the ear canal, and the greater the difference between the ultrasonic wave and the ultrasonic wave reflected signal. The greater the tightness level, the better the effect of forming the blocking effect, the higher the decibel of the audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction, the easier it is for the feedback microphone to collect the audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction, and the greater the signal strength of the ear canal audio signal collected by the feedback microphone.

[0085] The difference between the ultrasonic wave and the ultrasonic wave reflected signal and the tightness level can be determined according to a preset corresponding relationship, and the preset corresponding relationship can be determined according to actual needs. The difference between the ultrasonic wave and the ultrasonic wave reflected signal is not limited to specific differences, as long as the tightness level can be determined according to the two audio signals, it can be the difference between the energy or other differences.

[0086] In this embodiment, after the feedback microphone collects the ultrasonic reflection signal, the energy of the ultrasonic reflection signal can be calculated to obtain the ultrasonic reflection energy. The ultrasonic emission energy of the speaker emitting ultrasonic waves can be pre-stored, for example, pre-stored in a processor inside the headset, or can be determined when the speaker emits ultrasonic waves. The process of determining the ultrasonic emission energy and the ultrasonic reflection energy is not limited in this embodiment and can be used in subsequent embodiments.

[0087] The tightness level of the earphone can be determined based on the ultrasonic reflection energy and the ultrasonic emission energy of the speaker emitting the ultrasonic wave. For example, the tightness level is determined based on the energy difference between the ultrasonic reflection energy and the ultrasonic emission energy.

[0088] When ultrasonic waves are emitted through the speaker and the tightness level of the earphones is determined based on the ultrasonic waves and the ultrasonic reflection signals, the human ear is not very sensitive to ultrasonic waves and cannot hear them. Therefore, the tightness level is determined by ultrasonic waves without affecting the normal use of the user and causing no auditory interference to the user. The tightness level of the earphones can be determined without the user's perception, thereby reducing the impact on the user and improving the user's experience.

[0089] In another embodiment, the frequency of the ultrasonic wave is different from the frequency of the ear canal audio signal, so that the mutual influence between the ultrasonic wave and the ear canal audio signal can be reduced when determining the tightness level and collecting the ear canal audio signal. The frequency of the ultrasonic wave and the ultrasonic wave reflection signal can be the same. When determining the tightness level, the ultrasonic wave reflection signal can be filtered to filter out other audio signals other than the ultrasonic wave reflection signal frequency, thereby reducing the influence of other audio signals on the collection of the ultrasonic wave reflection signal, thereby reducing the influence on determining the tightness level of the earphone, and improving the accuracy of determining the tightness level of the earphone.

[0090] In another embodiment, calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy includes:

[0091] The ultrasonic reflection energy is obtained according to the energy of each sampling point in at least one signal frame of the ultrasonic reflection signal. The ultrasonic emission energy is obtained according to the energy of each sampling point in at least one signal frame of the ultrasonic reflection signal.

[0092] In one embodiment, the energy of a frame signal in the ultrasonic wave and the ultrasonic wave reflection signal can be determined by formula (1):

[0093]

[0094] Where L represents the frame length, k i represents the energy of the i-th sampling point in the frame signal. There are L sampling points in the frame length L. L can be determined according to actual needs, k i It can be determined after collection. The ultrasonic signal output by the speaker can be collected by a device with audio signal collection function in the headset, and the ultrasonic reflection signal can be collected by the feedback microphone. After collection, it can be transmitted to the corresponding processing device to determine k i , such as a controller.

[0095] The energy of a frame of ultrasonic signal can be determined by formula (2):

[0096]

[0097] x i Represents the energy of the i-th sampling point in a signal frame in ultrasound.

[0098] The energy of a frame of ultrasonic reflection signal can be determined by formula (3):

[0099]

[0100] y i Represents the energy of the i-th sampling point in a signal frame in the ultrasonic reflection signal.

[0101] In another embodiment, the ultrasonic emission energy may be pre-stored and may not be determined by the above formula (2).

[0102] In another embodiment, the tightness level may be determined based on the energy difference between ultrasonic emission energy and ultrasonic reflection energy. There is a preset mapping relationship between the energy difference and the tightness level. After the energy difference is determined, the tightness level may be determined based on the preset mapping relationship.

[0103] The energy difference can be expressed by formula (4):

[0104] ΔE=E1-E2 (4)

[0105] According to the preset mapping relationship between ΔE and tightness level, the tightness level can be determined after ΔE is determined. Table 1 is an example of a preset mapping relationship provided in this embodiment.

[0106] Table 1:

[0107] ΔE Tightness level More than 17dB 1 Greater than 14dB Less than 17dB 2 More than 11dB Less than 14dB 3 Greater than 8dB Less than 11dB 4 Less than 8dB 5

[0108] In Table 1, the smaller ΔE is and the larger the number corresponding to the level is, the higher the tightness level is, the better the blocking effect is, the higher the decibel of the audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal via bone conduction, the easier it is for the feedback microphone to collect the audio signal generated by transmitting the vibration generated by the interactive operation to the ear canal via bone conduction, and the greater the signal strength of the ear canal audio signal collected by the feedback microphone.

[0109] In another embodiment, the tightness level can also be determined based on the energy of one frame of the ultrasonic reflection signal and the energy of one frame of the ultrasonic wave. The tightness level can also be determined based on the energy of multiple frames of the ultrasonic reflection signal and the energy of multiple frames of the ultrasonic wave.

[0110] When determining the tightness level based on the energy of multiple signal frames in the ultrasonic wave and the energy of multiple signal frames in the ultrasonic reflection signal, the tightness level can be determined based on the energy difference between the average energy of the multiple signal frames in the ultrasonic wave and the average energy of the multiple signal frames in the ultrasonic reflection signal.

[0111] In another embodiment, step S100, collecting an ear canal audio signal in the ear canal by using a feedback microphone, includes:

[0112] When the ear canal audio signal collected by the feedback microphone includes ultrasonic waves output by the speaker of the earphone, filtering the ultrasonic waves to obtain a filtered ear canal audio signal;

[0113] and / or,

[0114] When the ear canal audio signal collected by the feedback microphone includes audio signals other than ultrasound and audio signals generated by transmitting vibrations generated by interactive operations to the ear canal through bone conduction, such as audio signals such as music, the ear canal audio signal is filtered to obtain a filtered ear canal audio signal.

[0115] After a certain degree of ear blocking effect is formed, when collecting ear canal audio signals, the speaker in the headset may output ultrasonic waves, and / or audio signals other than ultrasonic waves and audio signals generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction. The audio signals other than ultrasonic waves and audio signals generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction can be audio that the user normally needs, such as the audio of songs or videos. When the audio signals generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction, ultrasonic waves, and / or audio signals other than ultrasonic waves and audio signals generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction exist at the same time, the ultrasonic waves, and / or audio signals other than ultrasonic waves and audio signals generated by transmitting the vibration generated by the interactive operation to the ear canal by bone conduction can be filtered, so as to reduce the influence of other audio signals on the ear canal audio signals and improve the accuracy of collecting ear canal audio signals. Filtering the ear canal audio signals can be achieved by the controller of the headset or other devices with filtering functions, such as by echo cancellation algorithms.

[0116] In another implementation, when the tightness level of the earphone is low and there is a gap between the earphone and the ear canal, the audio signal in the external environment can also be transmitted to the ear canal, and the audio signal in the external environment is recorded as the ambient audio signal, which can be the audio signal in the environment where the earphone is located. At this time, the ear canal audio signal collected by the feedback microphone includes the ambient audio signal, and the ear canal audio signal is filtered to obtain the filtered ear canal audio signal, thereby reducing the influence of the ambient audio signal on the ear canal audio signal and improving the signal-to-noise ratio and accuracy of the collected ear canal audio signal.

[0117] In another embodiment, when the earphone is in a wearing state, the feedforward microphone is located outside the ear canal, and the ambient audio signal can be collected by the feedforward microphone in the earphone. Figure 2 B shown in .

[0118] In one embodiment, when the feedback microphone collects the ear canal audio signal, if the ear canal audio signal includes an ultrasonic signal, the ear canal audio signal may be filtered by low-pass filtering or the like.

[0119] In one embodiment, after the feedback microphone collects the ear canal audio signal, the controller in the headset can identify whether the ear canal audio signal includes ultrasound. Combined with the ambient noise collected by the feedforward microphone, it can identify whether the ear canal audio signal includes ambient noise, etc.

[0120] In another embodiment, the interactive operation recognition model is obtained by pre-training an initial neural network model through a training sample set using a machine learning method. The structure of the initial neural network model is not limited, and after being trained through a training sample set using a machine learning method, it can output a recognition result according to the audio feature to be recognized.

[0121] In one embodiment, the training sample set includes a positive sample set, and the positive sample set includes a plurality of positive samples;

[0122] Each positive sample includes: target interactive operation audio features, earphone tightness level and first label;

[0123] The target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs the target operation in the target area while wearing the earphone. The first label is used to identify the target operation and the tightness level of the earphone.

[0124] The target interactive operation audio feature can be an audio feature obtained by processing the sound signal in the ear canal collected by the feedback microphone and extracting the feature when the user performs the target operation in the target area. The target interactive operation audio feature in each positive sample and the tightness level of the headset in the current wearing state are used as the input of the initial neural network model, and the first label is used as the output of the initial neural network model. The number of positive samples in the positive sample set can be determined according to actual needs. The larger the number, the higher the accuracy of the interactive operation recognition model obtained by training.

[0125] The target operation performed by the user in the target area may at least include: single click, double click, or more consecutive clicks on the face area; single click, double click, or more consecutive clicks on the head area; single click, double click, or more consecutive clicks on the neck area. The target operation and target area may be determined according to actual usage requirements.

[0126] The target interactive operation audio feature may include a mel spectrum feature parameter and a mel cepstral feature parameter, and both the mel spectrum coefficient and the mel cepstral coefficient may be 40-dimensional coefficients. In different positive samples, the tightness level of the earphone in the wearing state is different, and / or the target interactive operation audio feature is different.

[0127] In another embodiment, the training sample set further includes a negative sample set, and the negative sample set includes a plurality of negative samples;

[0128] Each negative sample includes: non-target interactive operation audio features, earphone tightness level and second label;

[0129] The non-target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs non-target operations in the target area while wearing earphones; the second tag is used to identify the non-target operation and the tightness level of the earphone.

[0130] The non-target interactive operation audio feature can be an audio feature obtained by processing the sound signal in the ear canal collected by the feedback microphone and extracting the feature when the user performs a non-target operation in the target area. The non-target interactive operation audio feature in each negative sample and the tightness level of the earphone in the current wearing state are used as the input of the initial neural network model, and the second label is used as the output of the initial neural network model. The number of negative samples in the negative sample set can be determined according to actual needs. The more negative samples there are, the higher the accuracy of the interactive operation recognition model obtained by training.

[0131] The non-target operations performed by the user in the target area may at least include: single touch, double touch, and more continuous touches on the face area; single touch, double touch, and more continuous touches on the head area; single touch, double touch, and more continuous touches on the neck area; single tap, double tap, and more continuous taps on the face area, etc. The non-target operations and target areas may be determined according to actual usage requirements.

[0132] The non-target interaction operation audio features may include mel spectrum feature parameters and mel cepstral feature parameters, and both mel spectrum coefficients and mel cepstral coefficients may be 40-dimensional coefficients. In different negative samples, the tightness level of the earphone in the wearing state is different, and / or the non-target interaction operation audio features are different.

[0133] Non-target operations are different from target operations. By training the initial neural network model with positive sample sets and negative sample sets, the obtained interactive operation recognition model has higher accuracy, which can improve the accuracy of the output recognition results, thereby improving the accuracy of headphone control.

[0134] Combined with the tightness level of the headphones when worn, the initial neural network model is trained using positive and negative samples to obtain an interactive operation recognition model. Using the two-dimensional information of the Mel spectrum coefficient and Mel cepstral coefficient of the target audio signal, combined with the information of the tightness level of the headphones when worn, a total of three dimensions of information are used to train the network model, which can improve the recognition ability of the interactive operation recognition model and obtain more accurate recognition results.

[0135] In another embodiment, reference Figure 6 , is a schematic diagram of the structure of an earphone control device, wherein the earphone includes a feedback microphone, and the device includes:

[0136] The ear canal audio signal acquisition module 1 is used to collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal; wherein the ear canal audio signal includes: when the earphone is worn by the user, the user performs an interactive operation in the target area of ​​his / her head, and the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction.

[0137] An audio signal characteristic parameter determination module 2 is used to extract the characteristics of the ear canal audio signal to obtain audio signal characteristic parameters;

[0138] A tightness level determination module 3 is used to obtain the tightness level of the earphone in the current wearing state;

[0139] The to-be-recognized audio feature determination module 4 is used to generate the to-be-recognized audio feature according to the audio signal feature parameter and the tightness level;

[0140] A recognition result determination module 5, used for inputting the audio feature to be recognized into a preset interactive operation recognition model, and outputting a recognition result, wherein the recognition result includes an interactive operation identifier;

[0141] A control instruction determination module 6, used to determine a control instruction corresponding to the interactive operation identifier;

[0142] The control module 7 is used to control the playing state of the earphone according to the control instruction.

[0143] In another embodiment, the tightness level determination module is further used to detect the tightness of the earphone in the current wearing state at predetermined intervals to obtain the tightness level and store the tightness level.

[0144] In another embodiment, the earphone includes a speaker, and the tightness level determination module includes:

[0145] An ultrasonic reflection signal acquisition unit, used to acquire an ultrasonic reflection signal in the ear canal through the feedback microphone, wherein the ultrasonic reflection signal is an audio signal obtained after the ultrasonic wave is reflected by the ear canal when propagating in the ear canal, and the ultrasonic wave is emitted by the speaker at a predetermined frequency;

[0146] an ultrasonic reflection energy determination unit, used to calculate the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy;

[0147] The tightness level determination unit is used to obtain the tightness level according to the ultrasonic reflection energy and the pre-stored ultrasonic transmission energy.

[0148] In another embodiment, the ultrasonic reflection energy determination unit is further used for:

[0149] The ultrasonic reflection energy is obtained according to the energy of each sampling point in at least one signal frame in the ultrasonic reflection signal.

[0150] In another embodiment, the tightness level determination unit is further configured to:

[0151] The tightness level is determined according to an energy difference between the ultrasonic reflection energy and the ultrasonic transmission energy; wherein a preset mapping relationship exists between the energy difference and the tightness level.

[0152] In another embodiment, the apparatus further comprises:

[0153] The first filtering module is used to filter the ultrasonic reflection signal before calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy, so as to obtain the filtered ultrasonic reflection signal.

[0154] In another embodiment, the apparatus further comprises:

[0155] The second filtering module is used to perform low-pass filtering on the ear canal audio signal to obtain the filtered ear canal audio signal before extracting the feature of the ear canal audio signal to obtain the audio signal feature parameter.

[0156] In another embodiment, the interactive operation recognition model is obtained by pre-training an initial neural network model using a training sample set in a machine learning manner.

[0157] In another embodiment, the training sample set includes a positive sample set, and the positive sample set includes a plurality of positive samples;

[0158] Each of the positive samples includes: a target interactive operation audio feature, the tightness level and a first label;

[0159] The target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs a target operation in the target area while wearing earphones. The first label is used to identify the target operation and the tightness level.

[0160] In another embodiment, the training sample set further includes a negative sample set, and the negative sample set includes a plurality of negative samples;

[0161] Each of the negative samples includes: a non-target interactive operation audio feature, the tightness level and a second label;

[0162] The non-target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs non-target operations in the target area while wearing earphones; the second tag is used to identify the non-target operation and the tightness level.

[0163] In another embodiment, an electronic device is provided, including:

[0164] A processor and a memory for storing executable instructions capable of running on the processor, wherein:

[0165] When the processor is used to run the executable instructions, the executable instructions execute the method described in any one of the above embodiments.

[0166] In another embodiment, a non-transitory computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method described in any of the above embodiments is implemented.

[0167] It should be noted that the “first” and “second” in the embodiments of the present disclosure are only for the convenience of description and distinction and have no other specific meanings.

[0168] Figure 7 1 is a block diagram of a terminal device according to an exemplary embodiment. For example, the terminal device may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0169] Reference Figure 7 The terminal device may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0170] The processing component 802 generally controls the overall operation of the terminal device, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0171] The memory 804 is configured to store various types of data to support operations on the terminal device. Examples of such data include instructions for any application or method operating on the terminal device, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0172] The power component 806 provides power to various components of the terminal device. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device.

[0173] The multimedia component 808 includes a screen that provides an output interface between the terminal device and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the terminal device is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0174] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the terminal device is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0175] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0176] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the terminal device. For example, the sensor assembly 814 can detect the open / closed state of the terminal device, the relative positioning of components, such as the display and keypad of the terminal device, and the sensor assembly 814 can also detect the position change of the terminal device or a component of the terminal device, the presence or absence of user contact with the terminal device, the orientation or acceleration / deceleration of the terminal device, and the temperature change of the terminal device. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0177] The communication component 816 is configured to facilitate wired or wireless communication between the terminal device and other devices. The terminal device can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0178] In an exemplary embodiment, the terminal device may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0179] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0180] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for controlling a headset, characterized in that: The headset includes a feedback microphone, and the method includes: The feedback microphone collects the audio signal in the ear canal to obtain the ear canal audio signal; wherein the ear canal audio signal includes: when the earphone is worn by the user, the user performs an interactive operation on the target area of ​​his / her head, and the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction. Extracting features of the ear canal audio signal to obtain audio signal feature parameters; Acquire the tightness level of the headset in the current wearing state; the tightness level is determined according to the audio signal emitted by the speaker of the headset collected by the feedback microphone; Generating an audio feature to be identified according to the audio signal feature parameter and the tightness level; Inputting the audio feature to be identified into a preset interactive operation recognition model, and outputting a recognition result, wherein the recognition result includes an interactive operation identifier; Determining a control instruction corresponding to the interactive operation identifier; The playing state of the earphone is controlled according to the control instruction.

2. The method according to claim 1, characterized in that At predetermined intervals, the tightness of the earphone in the current wearing state is detected to obtain the tightness level, and the tightness level is stored.

3. The method according to claim 2, characterized in that The earphone includes the speaker, and detecting the tightness of the earphone in a current wearing state to obtain the tightness level includes: The feedback microphone is used to collect an ultrasonic reflection signal in the ear canal, wherein the ultrasonic reflection signal is an audio signal obtained after the ultrasonic wave is reflected by the ear canal when propagating in the ear canal, and the ultrasonic wave is emitted by the speaker at a predetermined frequency; Calculating the energy of the ultrasonic reflection signal to obtain ultrasonic reflection energy; The tightness level is obtained according to the ultrasonic reflection energy and the pre-stored ultrasonic emission energy.

4. The method according to claim 3, characterized in that The step of calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy includes: The ultrasonic reflection energy is obtained according to the energy of each sampling point in at least one signal frame in the ultrasonic reflection signal.

5. The method according to claim 3, characterized in that: The step of obtaining the tightness level according to the ultrasonic reflection energy and the pre-stored ultrasonic emission energy includes: The tightness level is determined according to an energy difference between the ultrasonic reflection energy and the ultrasonic transmission energy; wherein a preset mapping relationship exists between the energy difference and the tightness level.

6. The method according to claim 3, characterized in that Before calculating the energy of the ultrasonic reflection signal to obtain the ultrasonic reflection energy, the method further includes: The ultrasonic reflection signal is filtered to obtain the filtered ultrasonic reflection signal.

7. The method according to claim 1, characterized in that Before extracting features from the ear canal audio signal to obtain audio signal feature parameters, the method further includes: Perform low-pass filtering on the ear canal audio signal to obtain the filtered ear canal audio signal.

8. The method according to any one of claims 1 to 7, characterized in that: The interactive operation recognition model is obtained by pre-training an initial neural network model using a training sample set in a machine learning manner.

9. The method according to claim 8, characterized in that The training sample set includes a positive sample set, and the positive sample set includes a plurality of positive samples; Each of the positive samples includes: a target interactive operation audio feature, the tightness level and a first label; The target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs a target operation in the target area while wearing earphones. The first label is used to identify the target operation and the tightness level.

10. The method according to claim 8, characterized in that The training sample set also includes a negative sample set, and the negative sample set includes a plurality of negative samples; Each of the negative samples includes: a non-target interactive operation audio feature, the tightness level and a second label; The non-target interactive operation audio feature is an audio feature obtained by collecting sound signals in the ear canal through the earphone and processing the sound signals when the user performs non-target operations in the target area while wearing earphones; the second tag is used to identify the non-target operation and the tightness level.

11. An earphone control device, characterized in that: The headset includes a feedback microphone, and the device includes: An ear canal audio signal acquisition module, used to collect the audio signal in the ear canal through the feedback microphone to obtain the ear canal audio signal; wherein the ear canal audio signal includes: when the earphone is worn by the user, the user performs an interactive operation in the target area of ​​his / her head, and the vibration generated by the interactive operation is transmitted to the ear canal through bone conduction; An audio signal characteristic parameter determination module, used to extract characteristics of the ear canal audio signal to obtain audio signal characteristic parameters; A tightness level determination module, used to obtain the tightness level of the headset in the current wearing state; the tightness level is determined based on the audio signal emitted by the speaker of the headset collected by the feedback microphone; A module for determining audio features to be identified, used for generating audio features to be identified according to the audio signal feature parameters and the tightness level; A recognition result determination module, used for inputting the audio feature to be recognized into a preset interactive operation recognition model and outputting a recognition result, wherein the recognition result includes an interactive operation identifier; A control instruction determination module, used to determine the control instruction corresponding to the interactive operation identifier; The control module is used to control the playing state of the earphone according to the control instruction.

12. An electronic device, characterized in that: include: A processor and a memory for storing executable instructions capable of running on the processor, wherein: When the processor is used to run the executable instructions, the executable instructions execute the method described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images