Motion state detection method and apparatus, electronic device, storage medium, and product

Abstract

Embodiments of the present application provide a motion state detection method and device, electronic equipment, storage medium and product, aiming to solve the problem that the existing scheme simply relies on microphone acoustic signals, and step frequency extraction is unstable and motion state is easily misjudged in strong wind, complex noise and stationary interference scenes. The method comprehensively obtains environmental sound, ear canal vibration acoustic information and in-ear detection sensor raw sampling information, respectively representing wind noise intensity, acoustic step frequency and earphone and ear canal micro-contact change characteristics, and determines walking, running, cycling or stationary state based on multi-source information fusion. By introducing the micro-motion characteristics of the in-ear detection sensor and combining wind noise evaluation for collaborative judgment, the stability, accuracy and anti-interference ability of motion recognition in complex environments can be improved, and false step counting and state misrecognition can be reduced.

Description

Technical Field

[0001] This application relates to the field of motion detection in smart wearable devices, and more particularly to a method, apparatus, electronic device, storage medium, and product for motion state detection. Background Technology

[0002] Current smart wearable devices typically use the microphones of TWS (True Wireless Stereo) earphones to collect acoustic signals and identify the user's movement status.

[0003] However, detection schemes based on a single acoustic signal are susceptible to wind noise interference in outdoor scenarios, leading to a decrease in signal-to-noise ratio and unstable step frequency extraction. When stationary but with rhythmic external vibrations or low-frequency sounds, non-motion states may be identified as motion, which limits the detection accuracy and scene adaptability.

[0004] Therefore, improving the accuracy, stability, and anti-interference capabilities of TWS earphone motion detection in complex usage scenarios has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a motion state detection method, device, electronic device, storage medium, and product to solve the aforementioned technical problems. This solution addresses the need for headphone motion state recognition in complex usage scenarios. It comprehensively utilizes multi-dimensional information that characterizes external environmental interference, human movement rhythms, and changes in headphone wearing contact to collaboratively determine the user's current motion state, thereby improving the reliability and adaptability of motion state detection in different scenarios.

[0006] In a first aspect, embodiments of this application provide a motion state detection method, including:

[0007] It receives ambient sound signals collected by the first microphone, acoustic vibration signals inside the ear canal collected by the second microphone, and raw sampling signals collected by the in-ear detection sensor;

[0008] Micro-motion features are extracted from the original sampled signal; these features reflect the minute contact changes between the earphone and the ear canal caused by human movement.

[0009] When the micro-motion signal satisfies the periodicity condition, the acoustic step frequency is extracted from the acoustic vibration signal within the ear canal; wherein, the micro-motion signal is determined based on the micro-motion characteristics;

[0010] Determine the current wind noise intensity based on ambient sound signals;

[0011] The motion state is determined based on the current wind noise intensity, acoustic step frequency, and micro-motion characteristics.

[0012] In one possible embodiment, the micro-motion features include micro-motion step frequency and micro-motion amplitude. Extracting micro-motion features from the original sampled signal includes:

[0013] The original sampled signal is deDC processed to obtain an intermediate signal with the DC component removed;

[0014] The intermediate signal is bandpass filtered to obtain the micro-motion signal;

[0015] The micro-motion step frequency and micro-motion amplitude are calculated based on the micro-motion signal.

[0016] In one possible embodiment, the method further includes:

[0017] When the micro-motion signal does not meet the periodicity condition:

[0018] The micro-motion amplitude is calculated based on the micro-motion signal; if the micro-motion amplitude is less than the preset amplitude threshold, and the ambient sound signal has continuous low-frequency airflow energy in the preset frequency band, the current motion state is determined to be cycling.

[0019] If the conditions for riding are not met, the state is determined to be stationary.

[0020] In one possible embodiment, the acoustic step frequency is extracted from the acoustic vibration signal within the ear canal, including:

[0021] Bandpass filtering is applied to the acoustic vibration signal within the ear canal to match the bone conduction frequency range corresponding to footsteps;

[0022] The time interval for extracting signal energy peaks based on the filtered acoustic signal;

[0023] The acoustic step frequency is calculated based on the time interval.

[0024] In one possible embodiment, determining the current wind noise intensity based on ambient sound signals includes:

[0025] The ambient sound signal is subjected to low-pass filtering in a preset wind noise analysis frequency band, the root mean square energy value of the filtered signal is calculated, and the current wind noise intensity is obtained by quantization based on the root mean square energy value.

[0026] Accordingly, the motion state is determined based on the current wind noise intensity, acoustic cadence, and micro-motion characteristics, including:

[0027] Based on the current wind noise intensity, determine the first acceptance weight corresponding to the acoustic step frequency and the second acceptance weight corresponding to the micro-motion step frequency;

[0028] Based on the first and second acceptance weights, the acoustic step frequency and the micro-motion step frequency are weighted and fused to obtain the target step frequency;

[0029] The motion state is determined based on the target step frequency and micro-motion amplitude.

[0030] In one possible embodiment, the motion state is determined based on the target step frequency and the amplitude of the micro-motion, specifically including:

[0031] When the target step frequency is within the first preset step frequency range, the micro-motion amplitude is within the first preset amplitude range, and the energy of the filtered acoustic signal is less than the preset energy threshold, the current motion state is determined to be walking; the step frequency within the first preset step frequency range is greater than the preset amplitude threshold.

[0032] When the target cadence is within the second preset cadence range, the micro-motion amplitude is within the second preset amplitude range, and the energy of the filtered acoustic signal is greater than the preset energy threshold, the current movement state is determined to be running; wherein, the cadence within the second preset cadence range is not less than the cadence within the first preset cadence range, and the micro-motion amplitude within the second preset amplitude range is not less than the micro-motion amplitude within the first preset amplitude range.

[0033] In one possible embodiment, the method further includes: determining whether the micro-motion signal satisfies a periodic condition;

[0034] Determining whether a micro-motion signal meets the periodicity condition includes:

[0035] Extracting the time difference between adjacent peaks from the micro-motion signal;

[0036] Determine whether a preset number of consecutive time differences are within the first preset time difference range;

[0037] When a number of consecutive preset time differences are within the range of the first preset time difference, the micro-motion signal is determined to meet the periodicity condition.

[0038] Secondly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0039] The memory stores the instructions that the computer executes;

[0040] The processor executes computer execution instructions stored in memory, causing the processor to perform the methods described above.

[0041] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided above.

[0042] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method provided above.

[0043] By receiving ambient sound signals collected by the first microphone, acoustic vibration signals within the ear canal collected by the second microphone, and raw sampling signals collected by the in-ear detection sensor, and determining the current wind noise intensity based on the ambient sound signals, extracting micro-motion features from the raw sampling signals, and extracting acoustic step frequency from the acoustic vibration signals within the ear canal, the motion state is determined based on the current wind noise intensity, acoustic step frequency, and micro-motion features. This allows the system to simultaneously consider external wind noise interference, human body movement rhythm information, and minute contact changes between the earphone and the ear canal caused by human movement during the motion state determination process, thereby improving the accuracy, stability, and anti-interference capability of the earphone in motion state detection in complex scenarios. Attached Figure Description

[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0045] Figure 1 A schematic flowchart illustrating a motion state detection method provided in an embodiment of this application;

[0046] Figure 2 A flowchart illustrating another motion state detection method provided in this application embodiment;

[0047] Figure 3 A logical diagram illustrating a fusion decision provided in an embodiment of this application;

[0048] Figure 4 This is a schematic diagram of the structure of a motion state detection device provided in an embodiment of this application;

[0049] Figure 5 A schematic diagram of the structure of the electronic device provided in this application.

[0050] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0051] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0052] This application relates to motion state detection technology in smart wearable devices, applicable to the recognition of walking, running, cycling, and stationary states during daily wear of true wireless stereo headphones. The relevant system typically includes an outer microphone for acquiring ambient sound, an inner microphone for acquiring acoustic vibration signals within the ear canal, and an in-ear detection sensor for detecting the headphone wearing status.

[0053] Existing motion detection solutions mostly rely on acoustic signals collected by earphone microphones as the main basis for judgment. By analyzing the energy changes, peak intervals, or frequency characteristics of vibrations in the ear canal, the user's step frequency and motion state can be inferred. The basis of this type of method is that the vibrations generated when the human body walks or runs will form a detectable acoustic response through the ear canal and earphone structure.

[0054] However, in scenarios such as outdoor running and commuting cycling, ambient airflow and background noise can easily enter the microphone acquisition link, leading to a decrease in the signal-to-noise ratio of the acoustic vibration signal in the ear canal. This results in jumps, loss, or instability in gait frequency extraction, and a significant decrease in the accuracy of motion state recognition. Especially under strong wind conditions, a single acoustic feature is insufficient to continuously reflect real gait changes.

[0055] Meanwhile, even when the device is stationary, vehicle vibrations, low-frequency music, or other rhythmic external sounds can create periodic characteristics in the acoustic signal similar to walking frequency, causing the system to misidentify a non-motion state as walking or running, resulting in false step counts. Existing motion detection solutions struggle to balance stability and interference resistance in complex scenarios.

[0056] To address the aforementioned technical challenges, the original sampling signal from an in-ear detection sensor is used as a new source of motion characteristics. By utilizing the minute contact changes between the earphone and the ear canal caused by human movement, the information gaps in acoustic signals under conditions of strong winds and complex noise can be filled. Ambient sound can be used to determine the current wind noise intensity, the acoustic vibration signal within the ear canal can be used to extract the acoustic gait frequency, and the original sampling signal can be used to extract micro-motion features. By cross-referencing these three types of information, the basis for determining the motion state becomes more complete.

[0057] Based on this, the outer microphone, inner microphone, in-ear detection sensor and main control processing unit are used in the same detection link to comprehensively determine the acoustic step frequency and micro-motion characteristics according to the wind noise intensity. This can output more stable recognition results in usage scenarios such as walking, running, cycling and standing still, and alleviate the problem of misjudgment in complex environments.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0059] Figure 1 This is a flowchart illustrating a motion state detection method provided in an embodiment of this application. The execution entity in this embodiment can be any device with data processing capabilities. This application uses the main control processing unit in an earphone as an example for specific explanation. Figure 1 As shown in the embodiment of this application, a motion state detection method is provided, the method comprising:

[0060] S101: Receives ambient sound signals collected by the first microphone, acoustic vibration signals within the ear canal collected by the second microphone, and raw sampling signals collected by the in-ear detection sensor.

[0061] The first microphone in this step is used to collect ambient sound signals. It corresponds to the feedforward microphone located on the outside of the true wireless stereo earphones. Its target is external sound propagating through the air, as well as wind noise generated by airflow impacting the microphone. The ambient sound signal is the electrical signal or digital audio sampling sequence output by the first microphone, used to characterize changes in sound pressure and wind noise in the current external environment. The second microphone is used to collect acoustic vibration signals within the ear canal. It corresponds to the feedback microphone located inside the earphone facing the ear canal. Its target is the acoustic vibration response formed by the coupling of the ear canal, soft tissue, and earphone shell during human movement. The acoustic vibration signal within the ear canal is the acoustic sampling data output by the second microphone, reflecting bone conduction and contact vibration. The in-ear detection sensor is used to collect the raw sampling signal. This sensor is located at the earphone inlet and faces the ear canal contact area. It can be an infrared optical sensor or a capacitive sensor. The raw sampling signal is the basic data output by this sensor during continuous sampling, reflecting the minute contact changes between the earphone and the ear canal as the human body moves.

[0062] In specific implementation, the executing entity of this application can be the main control processing unit inside the earphone. The first microphone, the second microphone, and the in-ear detection sensor are respectively connected to the corresponding acquisition channels. The main control processing unit performs synchronous acquisition and time alignment processing on the received multi-channel signals to facilitate subsequent determination of wind noise intensity, acoustic cadence extraction, micro-motion feature extraction, and motion state determination. Furthermore, the received three-channel signals can first undergo buffer integrity checks and outlier removal before submitting valid data to subsequent steps. Based on the above analysis, this step completes the synchronous acquisition and unified input of three types of information: environmental dimension, ear canal acoustic dimension, and in-ear contact dimension, providing a data foundation for subsequent motion state determination.

[0063] S102: Extract micro-motion features from the original sampled signal; whereby the micro-motion features are used to reflect the minute contact changes between the earphone and the ear canal caused by human movement.

[0064] The raw sampling signal in this step comes from an in-ear detection sensor, which can be either an infrared optical sensor or a capacitive sensor. An infrared optical sensor obtains changes in the reflection from the ear canal surface by emitting and receiving infrared signals, while a capacitive sensor obtains changes in the contact state by detecting changes in the dielectric environment between the earphone and the ear canal. Both types of raw sampling signals output by the sensor contain a baseline value corresponding to the static wearing position and minute fluctuations caused by human movement. The micro-motion features extracted in this step are used to quantify this micro-motion information, enabling it to participate in motion state determination.

[0065] In practice, the main control processing unit processes the original sampled signal to obtain micro-motion features that characterize the contact changes between the earphone and the ear canal. In some implementations, the extracted micro-motion features can be used to reflect at least one of the rhythmic changes, amplitude of change, or stability during human movement. Considering changes in wearing tightness, differences in ear canal shape, and signal changes due to prolonged wear, this step can also perform adaptive processing on the micro-motion features to improve their comparability and stability under different usage conditions. This step is an important supplement to schemes that rely solely on acoustic information, because the in-ear detection sensor provides contact dimension information from a different physical source than airflow noise. Even under complex noise conditions, it can still retain contact change information reflecting human movement, thus providing an independent basis for subsequent state recognition.

[0066] S103: When the micro-motion signal meets the periodicity condition, the acoustic step frequency is extracted from the vibration acoustic signal in the ear canal; wherein, the micro-motion signal is determined based on the micro-motion characteristics.

[0067] In this step, the acoustic vibration signal within the ear canal is collected by a second microphone. This second microphone is located inside the earphone, close to the ear canal cavity, and can sense the acoustic vibration components coupled into the ear canal by footsteps, head movements, and bone conduction during walking or running. The acoustic gait frequency is a periodic gait index extracted from this acoustic vibration signal, used to reflect the rhythm of human movement. The acoustic gait frequency output in this step can be in continuous numerical form or accompanied by a confidence level for subsequent motion state determination.

[0068] In practical implementation, the main control processing unit can process the acoustic vibration signal within the ear canal when the micro-motion signal meets the periodicity condition to extract periodic information related to human movement and determine the acoustic gait frequency accordingly. Considering that wind noise, vehicle vibration, or other external interference may form a periodic response in the acoustic vibration signal within the ear canal during actual use, this step can judge the validity of the extraction results to improve the reliability of the acoustic gait frequency. Based on the above analysis, this step provides gait periodic information from the acoustic link within the ear canal, which can serve as one of the data sources for determining the motion state.

[0069] S104: Determine the current wind noise intensity based on ambient sound signals.

[0070] The current wind noise intensity in this step is used to characterize the degree to which airflow impacts the microphone on the outside of the headphones in the current environment. The ambient sound signal is collected by the first microphone, which is located on the outside of the headphones and is more sensitive to wind noise, road surface noise, and external traffic noise. Therefore, the level of wind noise interference can be characterized based on changes in the ambient sound signal. The wind noise intensity does not simply correspond to a single instantaneous amplitude, but is determined based on the overall variation characteristics of the ambient sound signal within a preset time range.

[0071] In specific implementation, the main control processing unit extracts the current analysis data from the ambient sound signal cached in S101 and processes it to determine the current wind noise intensity. In some implementations, the changes in the ambient sound signal over a continuous time range can also be combined to improve the stability of the wind noise intensity determination result. Based on the above analysis, this step is used to obtain the wind noise interference characterization information in the current environment, providing a basis for subsequent motion state determination. S105: Determine the motion state based on the current wind noise intensity, acoustic step frequency, and micro-motion characteristics.

[0072] The motion state in this step is the final output of this method, indicating whether the user is currently walking, running, cycling, or stationary. The current wind noise intensity comes from S104, the acoustic gait frequency from S103, and the micro-motion features from S102. The main control processing unit performs a joint determination based on the current wind noise intensity, acoustic gait frequency, and micro-motion features to obtain the final motion classification result.

[0073] In practice, the main control processing unit can combine the environmental interference level represented by the current wind noise intensity with the motion information corresponding to the acoustic gait frequency and micro-motion characteristics to identify the current user's motion state. Under different environmental conditions, the motion characteristics reflected by the acoustic gait frequency and micro-motion characteristics can be comprehensively utilized to improve the accuracy and stability of motion state determination.

[0074] This application provides a motion state detection method, including receiving ambient sound signals collected by a first microphone, acoustic vibration signals within the ear canal collected by a second microphone, and raw sampling signals collected by an in-ear detection sensor; extracting micro-motion features from the raw sampling signals, wherein the micro-motion features reflect the characteristics of minute contact changes between the earphone and the ear canal caused by human movement; extracting acoustic step frequency from the acoustic vibration signals within the ear canal when the micro-motion signals meet a periodicity condition; determining the current wind noise intensity based on the ambient sound signals; and determining the motion state based on the current wind noise intensity, acoustic step frequency, and micro-motion features. In this application, by incorporating the environmental dimension formed by the outer microphone, the ear canal acoustic dimension formed by the inner microphone, and the contact micro-motion dimension formed by the in-ear detection sensor into the same detection chain, the recognition of motion state is no longer limited to a single acoustic feature, thereby improving the applicability of motion state detection in complex environments.

[0075] In one possible implementation, the micro-motion features include micro-motion step frequency and micro-motion amplitude. Extracting the micro-motion features from the original sampled signal includes:

[0076] The original sampled signal is deDC processed to obtain an intermediate signal with the DC component removed;

[0077] The intermediate signal is bandpass filtered to obtain the micro-motion signal;

[0078] The micro-motion step frequency and micro-motion amplitude are calculated based on the micro-motion signal.

[0079] In practical implementation, the raw sampling signal collected by the in-ear detection sensor can first undergo mean elimination or baseline subtraction to obtain an intermediate signal with DC component removed. This intermediate signal is then filtered using a digital bandpass filter. The filter passband can be set according to the frequency range of the earphone's contact with micro-motions during walking or running, to preserve gait-related changes and reduce wind noise, random jitter, and static drift. For example, a 0.5-5Hz bandpass filter can be used. Subsequently, the micro-motion signal is periodically determined. When the interval between adjacent peaks, peak-valley repeatability, or autocorrelation characteristics meet preset conditions, the micro-motion step frequency is calculated based on the time difference between adjacent peaks, and the micro-motion amplitude is calculated based on the peak-to-peak value, peak-to-peak value, or effective amplitude. The digital bandpass filter can be implemented using FIR (Finite Impulse Response) or IIR (Infinite Impulse Response). In practical applications, other models can also be selected for this component, and this application does not limit this choice.

[0080] By first removing the DC component and then performing bandpass filtering, the micro-motion signal can more effectively reflect the changes in headphone contact caused by human movement, and output stable micro-motion step frequency and amplitude when the periodicity condition is met. Therefore, the micro-motion characteristics can serve as an independent basis for motion state determination, participating in recognition along with acoustic step frequency and wind noise intensity, thereby improving detection consistency and result stability in complex environments.

[0081] Based on the foregoing embodiments, the motion state detection method provided in this application further includes: when the micro-motion signal does not meet the periodicity condition: calculating the micro-motion amplitude based on the micro-motion signal; if the micro-motion amplitude is less than a preset amplitude threshold and the ambient sound signal has continuous low-frequency airflow energy in a preset frequency band, determining the current motion state as cycling; if the cycling determination condition is not met, determining it as a stationary state.

[0082] In actual processing, after receiving the micro-motion signal, the main control processing unit first judges its periodicity. If the judgment result is that the periodicity condition is not met, the main control processing unit calculates the amplitude envelope, peak-to-peak value, or equivalent amplitude of the micro-motion signal to obtain the micro-motion amplitude, and compares this micro-motion amplitude with a preset amplitude threshold. Simultaneously, the main control processing unit performs frequency domain analysis on the ambient sound signal collected by the first microphone, extracts continuous low-frequency airflow energy within a preset frequency band, and detects whether this energy meets the duration requirement. When the micro-motion amplitude is less than the preset amplitude threshold, and the ambient sound signal simultaneously meets the aforementioned low-frequency airflow characteristics, the riding state is output; for example, the preset frequency band can be 50-200Hz. When any condition is not met, the stationary state is output.

[0083] This judgment logic combines the amplitude of micro-motions with the low-frequency airflow characteristics in the ambient sound, enabling it to distinguish between cycling and stationary states even when the micro-motion signal lacks periodicity. For situations where the wearer is cycling but the headphone contact changes are minimal, the system can assist in identification through continuous low-frequency airflow energy. Conversely, for stationary states with sporadic vibrations or short-term environmental interference, the system can determine the user as stationary because the cycling condition is not met, thus making the output for motion states more stable.

[0084] In one possible implementation, the acoustic step frequency is extracted from the acoustic vibration signal within the ear canal, including:

[0085] Bandpass filtering is applied to the acoustic vibration signal within the ear canal to match the bone conduction frequency range corresponding to footsteps;

[0086] The time interval for extracting signal energy peaks based on the filtered acoustic signal;

[0087] The acoustic step frequency is calculated based on the time interval.

[0088] In practical implementation, the acoustic vibration signal within the ear canal is first processed by a digital bandpass filter. The passband of the digital bandpass filter covers the bone conduction frequency range corresponding to footsteps. The lower limit of the passband can be set to tens of hertz, and the upper limit can be set to hundreds of hertz. The digital bandpass filter can be implemented using an FIR or IIR structure to retain the energy components related to footsteps and suppress environmental noise and unrelated low-frequency drift. For example, a 20-200Hz bandpass filter can be used. Subsequently, envelope extraction or short-time energy calculation is performed on the filtered acoustic signal, and local maxima of energy are searched within a continuous time window to obtain the time interval between adjacent energy peaks. Based on the average or median of multiple time intervals, the acoustic gait frequency can be calculated, which can be expressed as the number of corresponding footstep events per unit time.

[0089] During operation, the system first determines whether to enable the acoustic gait extraction link based on whether the micro-motion signal meets the periodicity condition. When the condition is met, the system performs frequency band constraints and peak interval calculations on the acoustic vibration signal in the ear canal so that the extraction result retains only the periodic information synchronized with the actual gait.

[0090] This method bases the calculation of acoustic gait frequency on micro-motion signal verification and determines it jointly by limiting the bone conduction frequency band and the energy peak interval, thereby improving the stability of the gait frequency output. Since both filtering and peak interval calculation are directed towards the effective frequency band corresponding to footsteps, the obtained acoustic gait frequency has a clearer correspondence with the actual movement rhythm, providing reliable input for subsequent movement state determination.

[0091] In one possible implementation, determining the current wind noise intensity based on the ambient sound signal includes: performing low-pass filtering on the ambient sound signal within a preset wind noise analysis frequency band, calculating the root mean square energy value of the filtered signal, and quantizing the current wind noise intensity based on the root mean square energy value; correspondingly, determining the motion state based on the current wind noise intensity, acoustic step frequency, and micro-motion characteristics includes: determining a first acceptance weight corresponding to the acoustic step frequency and a second acceptance weight corresponding to the micro-motion step frequency based on the current wind noise intensity; performing weighted fusion of the acoustic step frequency and the micro-motion step frequency based on the first acceptance weight and the second acceptance weight to obtain the target step frequency; and determining the motion state based on the target step frequency and the micro-motion amplitude.

[0092] In actual processing, the ambient sound signal collected by the external microphone first enters the low-pass filter module, which filters out higher-frequency background components and outputs a wind noise candidate signal. For example, the ambient sound signal can be low-pass filtered from 50 to 500 Hz. Then, the candidate signal is windowed and sampled, and the root mean square energy value is calculated. The main control processing unit converts this energy value into the current wind noise intensity according to a pre-calibrated mapping relationship. Subsequently, the main control processing unit obtains the first and second acceptance weights by looking up a table or calculating based on the wind noise intensity. Under low wind noise conditions, the proportion of acoustic gait frequency is increased, and under high wind noise conditions, the proportion of micro-motion gait frequency is increased. Then, the two types of gait frequencies are weighted and fused according to their weights to form the target gait frequency. The target gait frequency and micro-motion amplitude are then input into the state classification model or threshold judgment logic to output the walking, running, cycling, or stationary state.

[0093] This method fuses environmental wind noise and step frequency into the same decision chain, allowing the weights to adaptively adjust with wind noise intensity. This maintains the stability of the target step frequency even under strong winds or complex noise conditions. Furthermore, it supplements the verification of the target step frequency through micro-motion amplitude, reducing misjudgments caused by single acoustic features. As a result, the continuity and anti-interference capability of motion state recognition results are improved.

[0094] In one possible implementation, the motion state is determined based on the target cadence and the micro-motion amplitude, specifically including: when the target cadence is within a first preset cadence range, the micro-motion amplitude is within a first preset amplitude range, and the energy of the filtered acoustic signal is less than a preset energy threshold, the current motion state is determined to be walking; the amplitude within the first preset amplitude range is greater than the preset amplitude threshold; when the target cadence is within a second preset cadence range, the micro-motion amplitude is within a second preset amplitude range, and the energy of the filtered acoustic signal is greater than the preset energy threshold, the current motion state is determined to be running; wherein, the cadence within the second preset cadence range is not less than the cadence within the first preset cadence range, and the micro-motion amplitude within the second preset amplitude range is not less than the micro-motion amplitude within the first preset amplitude range.

[0095] In its implementation, after obtaining the target step frequency and micro-motion amplitude, the system compares the target step frequency with a preset step frequency range and the micro-motion amplitude with a preset amplitude range, and calculates the energy value of the filtered acoustic signal. If the target step frequency falls within the first preset step frequency range and the micro-motion amplitude falls within the first preset amplitude range, and the energy value is less than a preset energy threshold, then the system outputs a walking state. If the target step frequency falls within the second preset step frequency range and the micro-motion amplitude falls within the second preset amplitude range, and the energy value is greater than the preset energy threshold, then the system outputs a running state. The first preset step frequency range corresponds to lower-intensity periodic movement, the second preset step frequency range corresponds to higher-intensity periodic movement, and the second preset amplitude range and the second preset step frequency range together characterize more significant changes in headphone micro-motion.

[0096] In this judgment process, the target gait frequency is set to balance the contributions of acoustic gait frequency and micro-motion gait frequency under different wind noise conditions, while the micro-motion amplitude is set to supplement the motion discrimination criteria when acoustic energy is insufficient. Walking and running judgments rely on different gait frequency ranges, amplitude ranges, and acoustic energy thresholds, respectively, thus allowing different exercise intensities to correspond to different classification conditions. This scheme, by jointly constraining gait frequency, micro-motion amplitude, and acoustic energy, ensures that the motion state output remains consistent with the actual gait changes of the human body.

[0097] With this method, the determination of walking and running no longer relies solely on a single acoustic feature, but rather combines the target step frequency, micro-motion amplitude, and filter energy for joint identification. This makes the state classification results more stable and has a stronger adaptability to individual differences and environmental interference during headphone wearing.

[0098] Based on the foregoing embodiments, the motion state detection method provided in this application further includes: determining whether a micro-motion signal satisfies a periodicity condition; determining whether a micro-motion signal satisfies a periodicity condition includes: extracting the time difference between adjacent peaks from the micro-motion signal; determining whether a consecutive preset number of time differences are within a first preset time difference range; and determining that the micro-motion signal satisfies a periodicity condition when a consecutive preset number of time differences are within the first preset time difference range. When the micro-motion signal satisfies the periodicity condition, the micro-motion step frequency and micro-motion amplitude are extracted based on the periodicity result and used as input features for subsequent motion state determination.

[0099] In practical implementation, after receiving the micro-motion signal, the main control processing unit first performs peak search on the signal to obtain the sampling point position corresponding to each peak, and then calculates the time difference sequence based on the position difference between adjacent peaks and the sampling period. Subsequently, the time difference sequence is compared with a first preset time difference range. If a preset number of time differences continuously fall within this range, a determination result indicating periodicity is met is output; if any time difference exceeds the range, the periodicity is not met. The first preset time difference range can be pre-configured according to the earphone wearing position, sampling rate, and target motion rhythm, and other parameters can also be selected in practical applications, which are not limited in this application.

[0100] This judgment method uses the consistency of the intervals between multiple consecutive adjacent wave peaks as a basis, which can distinguish between short-term jitter, occasional contact, and irregular vibration, and makes the periodic judgment results more consistent with the actual gait rhythm. When the micro-motion signal is determined to meet the periodic condition, the system can further use the signal to calculate the micro-motion step frequency and micro-motion amplitude, thereby providing a stable input for subsequent motion state fusion judgment.

[0101] With this approach, the periodicity of micro-motion signals no longer depends on a single peak interval, but is constrained by the consistency of continuous time difference, making the periodic feature extraction results more stable, thereby improving the reliability of micro-motion step frequency extraction and providing support for accurate identification of motion state under different wind noise conditions.

[0102] This application discloses a motion state detection method. During operation, it receives ambient sound signals collected by a first microphone, acoustic vibration signals within the ear canal collected by a second microphone, and raw sampling signals collected by an in-ear detection sensor. First, the ambient sound signals are low-pass filtered in a preset wind noise analysis frequency band. The root mean square energy value of the filtered signal is calculated, and the current wind noise intensity is quantified based on the root mean square energy value. Simultaneously, the raw sampling signals are de-DC processed to obtain an intermediate signal with the DC component removed. This intermediate signal is then band-pass filtered to obtain a micro-motion signal reflecting the minute contact changes between the earphone and the ear canal caused by human movement.

[0103] In one possible implementation, the time difference between adjacent peaks is extracted from the micro-motion signal, and it is determined whether a preset number of consecutive time differences fall within a first preset time difference range. When a preset number of consecutive time differences fall within the first preset time difference range, the micro-motion signal is determined to meet the periodicity condition. When the micro-motion signal meets the periodicity condition, the micro-motion step frequency and micro-motion amplitude are calculated based on the micro-motion signal, and the acoustic vibration signal within the ear canal is bandpass filtered to match the bone conduction frequency range corresponding to footsteps. Subsequently, the time interval of the signal energy peaks is extracted based on the filtered acoustic signal, and the acoustic step frequency is calculated based on the time interval. Then, based on the current wind noise intensity, the first acceptance weight corresponding to the acoustic step frequency and the second acceptance weight corresponding to the micro-motion step frequency are determined. The acoustic step frequency and the micro-motion step frequency are weighted and fused using the first acceptance weight and the second acceptance weight to obtain the target step frequency.

[0104] After obtaining the target step frequency, the current movement state is determined by combining the micro-motion amplitude and the filtered acoustic signal energy. When the target step frequency is within a first preset step frequency range, the micro-motion amplitude is within a first preset amplitude range, and the filtered acoustic signal energy is less than a preset energy threshold, the current movement state is determined to be walking, where the step frequency within the first preset step frequency range is greater than the preset amplitude threshold. When the target step frequency is within a second preset step frequency range, the micro-motion amplitude is within a second preset amplitude range, and the filtered acoustic signal energy is greater than the preset energy threshold, the current movement state is determined to be running, where the step frequency within the second preset step frequency range is not less than the step frequency within the first preset step frequency range, and the micro-motion amplitude within the second preset amplitude range is not less than the micro-motion amplitude within the first preset amplitude range.

[0105] When the micro-motion signal does not meet the periodicity condition, the micro-motion step frequency is not calculated, but the micro-motion amplitude is calculated based on the micro-motion signal. If the micro-motion amplitude is less than the preset amplitude threshold and there is continuous low-frequency airflow energy in the preset frequency band in the ambient sound signal, the current motion state is determined to be cycling. If the micro-motion amplitude is not less than the preset amplitude threshold, or there is no continuous low-frequency airflow energy in the preset frequency band in the ambient sound signal, the current motion state is determined to be stationary.

[0106] The core logic of this application will be explained again below:

[0107] The system simultaneously collects three signals: ambient sound signal from the outside of the earphone (FFMic, Feed-Forward Microphone), acoustic vibration signal from the inside of the ear canal (FBMic, Feed-Back Microphone), and raw data from the in-ear detection sensor (Raw Data), i.e., the raw sampling signal in the above embodiment. The system extracts the micro-motion features from the raw data from the wear sensing signal and performs weighted fusion with the acoustic step frequency to determine the final motion state.

[0108] Specific steps:

[0109] Step 1: Multidimensional signal acquisition

[0110] Acoustic channel: FF microphone is used to collect ambient sound signals, and FB microphone is used to collect vibration acoustic signals in the ear canal.

[0111] Non-acoustic channel: Acquires the raw sampled signal (ADC value) from the in-ear detection sensor (such as an infrared optical sensor or a capacitive sensor), rather than simply acquiring the "worn / not worn" on / off state. Here, ADC stands for Digital-to-Analog Converter.

[0112] Step 2: Signal Preprocessing

[0113] Bandpass filtering is applied to the acoustic vibration signal within the ear canal to match the frequency range of bone conduction of footsteps (20-200Hz).

[0114] The raw signal from the in-ear detection sensor is subjected to DC removal and low-pass / band-pass filtering (0.5-5Hz) to remove the static reference value of the sensor itself, and only retain the small fluctuation component caused by motion, i.e., the micro-motion signal.

[0115] Step 3: Feature Extraction

[0116] Acoustic characteristics: Extract the energy peak interval of the filtered acoustic vibration signal in the ear canal and calculate the acoustic step frequency.

[0117] Micro-motion characteristics: Perform periodic analysis on the micro-motion signal (such as autocorrelation analysis or zero-crossing detection). If the periodicity is satisfied, extract the micro-motion step frequency and micro-motion amplitude. If the periodicity is not satisfied, extract the micro-motion amplitude.

[0118] Environmental characteristics: The current wind noise level is determined using an FF microphone (root mean square energy is calculated after low-pass filtering of 50-500Hz).

[0119] Step 4: Modal fusion decision (core strategy)

[0120] Scenario A (Low wind noise / Indoor): The acoustic step frequency of the FB microphone is the primary source, supplemented by micro-motion signals. The two are cross-checked to improve accuracy.

[0121] Scenario B (High Wind Noise / Cycling): The microphone signal-to-noise ratio deteriorates. In this case, the weight of acoustic cadence is automatically reduced, while the weight of micro-motion signals is increased. Because the in-ear sensor uses light / electrical signals, it is completely unaffected by wind interference, and its periodic fluctuations remain accurate.

[0122] Scenario C (Stationary to prevent false judgment): If the FB microphone detects a rhythmic sound similar to footsteps (such as music), but there is no micro-motion signal (no fluctuation), it is judged as "pseudo-motion" and is not counted as a step.

[0123] Step 5: Classify exercise patterns by combining the characteristics of the three to distinguish walking, running, and cycling:

[0124] Walking: Step frequency 40-100 steps / minute, micro-motion amplitude is threshold ≤ amplitude < 2 times threshold, FB ear canal vibration acoustic signal energy is low.

[0125] Running: cadence of 100-200 steps / minute, micro-movement amplitude ≥ 2 times the threshold, and high level of acoustic signal energy of FB ear canal vibration.

[0126] Cycling: There is no stable periodic micro-motion step frequency (although the filtered signal fluctuates, the peak time difference δT does not meet the periodic condition of "three or more consecutive stable steps within 0.3s-1.5s"). The micro-motion amplitude is less than the threshold, but there are non-periodic low-frequency random micro-motions (after the road bumps are buffered by the soft tissue of the ear canal, they appear as irregular fluctuations in the 0.5-5Hz frequency band, which is different from the strong periodicity of gait). The FF microphone can detect the ambient airflow sound (50-200Hz continuous low-frequency energy) during cycling. Combining the above non-periodic micro-motion and airflow sound characteristics, it is determined to be a cycling state, and no steps are counted.

[0127] Figure 2 A flowchart illustrating another motion state detection method provided in this application embodiment is shown below. Figure 2 The diagram illustrates the complete workflow of the motion state detection method in this application, with the core logic as follows:

[0128] 1. Trigger Startup: After the headphones detect that the user is wearing the headphones, they automatically start the motion detection process;

[0129] 2. Synchronous acquisition: The main control chip synchronously acquires the raw sampling signal from the in-ear detection sensor and the dual-channel acoustic signal from the feedforward / feedback microphone to ensure the alignment of timestamps of multi-source data;

[0130] 3. Parallel processing: The original sampled signal is detrended, filtered, and peak detected to extract micro-motion features; the acoustic signal is filtered and energy is calculated to extract acoustic step frequency and wind noise features.

[0131] 4. Fusion Decision: Based on the wind noise intensity, the fusion weights of the two types of features are dynamically adjusted to complete the core determination of motion state, while realizing the prevention of false motion.

[0132] 5. Mode Classification and Output: Combining features such as cadence and micro-motion amplitude, the exercise modes such as walking, running, and cycling are further subdivided, and the data such as exercise status, cadence, and steps are finally output to the terminal.

[0133] This diagram clearly illustrates the entire chain logic of "wearing trigger - multi-source acquisition - parallel processing - fusion decision - classification output", intuitively demonstrating the core innovation of this application based on the fusion of raw data from wearable sensors and acoustic signals.

[0134] Figure 3 This application provides a logical diagram illustrating a fusion decision mechanism, as shown in the embodiments below. Figure 3 As shown, the core multimodal dynamic weighted fusion decision-making process of this application is demonstrated. It is the core logic for achieving highly robust motion state detection across all scenarios and is divided into three parallel links:

[0135] 1. Wind noise scene determination and weight generation chain

[0136] Wind noise energy calculation module: performs 0-500Hz low-pass filtering on the ambient sound signal collected by the feedforward microphone, calculates the root mean square energy value of the signal, and quantifies the wind noise intensity of the current environment.

[0137] Scene determination module: Based on the preset wind noise energy threshold, the usage scene is divided into low wind noise scene (wind level ≤ 3) and high wind noise scene (wind level ≥ 4).

[0138] Weighted weight determination module: Dynamic output fusion weight: In low wind noise environment, acoustic step frequency weight is 0.7 and micro-motion step frequency weight is 0.3; in high wind noise environment, acoustic step frequency weight is 0 and micro-motion step frequency weight is 1.0.

[0139] 2. Dual-path feature extraction and preprocessing link

[0140] Acoustic gait frequency characteristic link: The acoustic gait frequency is calculated by extracting the time interval of the signal energy peak after preprocessing such as noise reduction and filtering, using the bone conduction acoustic signal collected by the FB microphone in the ear canal as input;

[0141] Micro-motion step frequency characteristic link: Taking the original sampling signal of the in-ear detection sensor as input, after preprocessing such as detrending, 0.5-5Hz bandpass filtering, and peak detection, the time interval of the effective peak is extracted and the micro-motion step frequency is calculated.

[0142] 3. Dynamic weighted fusion and result output link

[0143] Dynamic weighted fusion module: Receives the weights from the wind noise scene link output, as well as the acoustic cadence and micro-motion cadence from the dual-path feature link output, and performs weighted calculations according to the corresponding weights to generate the target cadence; at the same time, it has built-in anti-false judgment logic: when the headphones are worn properly, if only the acoustic cadence is detected and there is no effective micro-motion cadence, it is directly judged as pseudo-motion and outputs a static state.

[0144] The results output module outputs the fused target step frequency and motion state determination results for subsequent motion mode classification and terminal data synchronization.

[0145] The solution of this application will be described in detail below with specific embodiments.

[0146] This embodiment provides a detailed description of motion detection scenarios for TWS earphones. This solution can also be directly extended to adapt to various smart earphone products equipped with microphones and wear detection sensors, such as OWS (Open Wearable Stereo), neckband, and over-ear earphones. The hardware foundation of the TWS earphones in this embodiment is: a main control chip (such as a Bluetooth audio main control SOC, where SOC stands for System-on-Chip), equivalent to the main control processing unit in the previous embodiment, one or more feedforward (FF) microphones, one or more feedback (FB) microphones, and an in-ear detection sensor (either an infrared optical sensor or a capacitive sensor, or a combination thereof). No additional motion detection hardware such as an IMU (Inertial Measurement Unit) or gyroscope is added; all detection is performed based on the earphone's existing hardware.

[0147] (I) Hardware Basics and Parameter Settings

[0148] 1. Sampling parameters: The audio sampling rate of the FF / FB microphone is reduced to 16KHz in motion detection scenarios (in non-motion scenarios, the sampling rate is restored to the normal ANC (Active Noise Cancellation) / call mode to reduce the overall power consumption during motion detection).

[0149] 2. Sensor Deployment: The FF microphone is deployed on the outer cavity of the earphone to collect external acoustic signals such as ambient sound signals and wind noise; the FB microphone is deployed on the inner side of the earphone close to the ear canal to collect internal acoustic signals such as footsteps and vibrations transmitted by bone conduction during human movement; the in-ear detection sensor is deployed at the ear canal where the earphone enters the ear to ensure that the sensor can accurately capture the minute relative displacement / deformation between the earphone and the ear canal.

[0150] 3. Threshold preset: Based on a large amount of real-world human motion measurement data, core judgment thresholds are preset, which can support slight calibration on the user side.

[0151] The first preset time difference range is 0.3s-1.5s (corresponding to a cadence of 40-200 steps / minute, covering the normal cadence range for walking and running).

[0152] Micro-motion peak-to-peak threshold: Distinguish between motion micro-motion and inherent sensor noise, and adjust according to the actual sensor sensitivity, etc.

[0153] Wind noise energy threshold: Low threshold E Threshold−Low (Corresponding to ambient wind level ≤ 3, no wind / light breeze), high threshold E Threshold−High (Corresponding to an ambient wind level ≥ 4, strong wind), the determination is based on the energy value of the ambient sound signal collected by the FF microphone.

[0154] Wind noise level calibration test specifications:

[0155] Test environment: Standard wind tunnel test chamber was used, with wind speed gradients of 2m / s to 12m / s segmented calibration (corresponding to wind noise levels 1 to 5).

[0156] Calibration method: Fix the prototype in the wind tunnel fixture, collect the raw energy of the MIC at three blowing angles: front, side and oblique. Take the average of multiple test samples as the classification threshold of the model.

[0157] The motion state detection in this application is a real-time online detection. After the main control chip is powered on and detects the "wearing state", the detection process is automatically started. There is no manual operation by the user throughout the process. It is divided into six steps: data reading, raw sampling signal processing, acoustic signal processing, fusion decision, motion mode classification, and result output. Each step is executed synchronously and in parallel, with a detection cycle of 200ms, ensuring the real-time nature of motion state determination.

[0158] The reason for choosing a detection period of 200ms is as follows: If we back-calculate the sampling period based on the lowest step frequency of 40 steps / minute (single step period ≈ 1500ms), it seems that 200ms is too redundant. However, in high step frequency scenarios, the Nyquist sampling theorem must be followed: the sampling frequency must be greater than twice the frequency of the measured motion. If the sampling period is set too long or the sampling rate is insufficient, the high-frequency step signal will be aliased and distorted, resulting in missed steps and incorrect step frequency calculation. Therefore, we prioritize the extreme high step frequency as the basis for selecting the sampling period, rather than using only the low frequency lower limit as a constraint. Finally, a 200ms refresh period was selected.

[0159] Step 1: Data Reading

[0160] The main control chip continuously reads the original sampling signal from the in-ear detection sensor (ADC voltage value for the infrared sensor and ADC value after capacitance conversion) at a fixed sampling rate of 100Hz through its own interface, and buffers the original sampling signal in a sliding window in 2-second units. At the same time, it synchronously acquires the sampling acoustic data from the FF and FB microphones through the audio acquisition interface and stores them in independent acoustic data buffers to ensure that the timestamps of non-acoustic signals and acoustic signals are synchronized without time difference offset.

[0161] Step 2: Raw Sampling Signal Processing

[0162] The raw sampling signals from the cached in-ear detection sensor undergo multi-dimensional preprocessing and feature extraction, retaining only periodic micro-motion features related to human movement. Specific operations include:

[0163] 1. Detrending processing: A linear fitting algorithm is used to process the original sampled signal within a 2-second sliding window to eliminate baseline drift caused by slow changes in wearing tightness and slight body position shift of the headphones, thus ensuring the stability of the data baseline;

[0164] 2. Filtering: The detrended data is subjected to a 0.5-5Hz bandpass filter to remove the sensor’s inherent high-frequency noise (>5Hz) and DC component (<0.5Hz), retaining only the micro-motion signal that matches the gait frequency;

[0165] 3. Peak detection: An adaptive peak detection algorithm is used to identify peaks in the micro-motion signal and remove false peaks caused by accidental jitter (peak duration <100ms is judged as a false peak).

[0166] 4. Micro-motion characteristic determination: Calculate the time difference δT between effective peaks. If three or more consecutive δT values ​​are stable between 0.3s and 1.5s, and the peak-to-peak value exceeds the preset micro-motion threshold, then the periodicity condition is met, and the step frequency (micro-motion) and micro-motion amplitude are recorded. If the above conditions are not met, then the periodicity condition is not met. In particular, when the signal fluctuates and some δT values ​​fall within the 0.3s-1.5s range, but the number of consecutive effective cycles is less than three or the peak time difference shows irregular jumps, the periodicity condition is still not met. This feature is used to distinguish motion states such as cycling that have low-frequency random vibrations but irregular gait.

[0167] Step 3: Acoustic Signal Processing

[0168] The acoustic data from the FF and FB microphones are processed separately to extract wind noise features and acoustic step frequencies. Specific steps are as follows:

[0169] 1. Acoustic gait frequency extraction using FB microphone: The acoustic vibration signal in the ear canal acquired by the FB microphone is bandpass filtered at 20-200Hz (matching the frequency range of bone conduction of footsteps), the energy peak of the filtered signal is calculated, and the time difference between the energy peaks is extracted to obtain the acoustic gait frequency;

[0170] 2. FF microphone wind noise energy calculation: The ambient sound signal collected by the FF microphone is low-pass filtered from 50 to 500 Hz (wind noise is mainly concentrated in the low frequency band), and the root mean square energy value of the filtered signal is calculated to obtain the wind noise energy E_wind;

[0171] 3. Preliminary judgment of the validity of acoustic features: If E_wind < E_Threshold-Low, then the acoustic step frequency is determined to be a valid feature; if E_wind ≥ E_Threshold-High, then the acoustic step frequency is determined to be an invalid feature.

[0172] Step 4: Integrated Decision Making

[0173] The fusion decision prioritizes the execution of the anti-misjudgment verification logic, then divides the scene based on the wind noise energy E_wind, and combines the characteristics and effectiveness of acoustic step frequency and micro-motion step frequency to complete dynamic weighted fusion and motion state determination.

[0174] Step 1: Prioritize the execution of the anti-false positive check (this check has higher priority than the wind noise scene check and is not affected by the wind noise intensity).

[0175] If the second microphone detects a valid acoustic step frequency, but the in-ear detection sensor does not detect a valid micro-motion feature, including the absence of periodic fluctuations in the micro-motion signal or a micro-motion fluctuation amplitude below a set threshold, then it is determined to be a false motion state, and a static state is forcibly output, with no step count result output. This logic is used to avoid false step counts caused by interference from vehicle bumps and vibrations, external rhythmic music, low-frequency noise, etc.

[0176] Only after passing the above-mentioned anti-misjudgment verification, that is, under the premise that the acoustic step frequency and the effective micro-motion feature exist simultaneously, or only the effective micro-motion feature exists, can the wind noise scene be divided according to the wind noise energy E_wind, and the final step frequency be calculated by dynamic weighted fusion.

[0177] Scenario 1: Low wind noise environment (E_wind < E_Threshold - Low)

[0178] The acoustic signal is less susceptible to interference and has a good signal-to-noise ratio. The acoustic step frequency is set as the main weight (e.g., 0.7) and the micro-motion step frequency is set as the secondary weight (e.g., 0.3). The candidate step frequency is obtained by weighted averaging.

[0179] If the deviation between the acoustic step frequency and the micro-motion step frequency is ≤10 steps / minute, the weighted candidate step frequency is directly output as the target step frequency;

[0180] If the deviation between the two is greater than 10 steps / minute, it is determined to be an occasional error of a single sensor, and the median of the acoustic step frequency and the micro-motion step frequency is taken as the target step frequency.

[0181] Scenario 2: High wind noise environment (E_wind≥E_Threshold-High)

[0182] Acoustic signals are severely interfered with by wind noise, resulting in a deteriorated signal-to-noise ratio. The acoustic step frequency weight is reduced (e.g., 0.2), and the micro-motion step frequency weight is increased (e.g., 0.8). The reliability of step counting results under high wind noise conditions is ensured by relying on the micro-motion step frequency output by the in-ear detection sensor, which is not affected by wind noise. The final step frequency is obtained by weighted calculation.

[0183] Note: The weights 0.7 / 0.3 and 0.2 / 0.8 are exemplary values ​​and can be adaptively adjusted according to the hardware calibration.

[0184] Step 5: Classification of Movement Patterns

[0185] Based on the final step frequency, and combining two characteristics—micro-motion amplitude and acoustic signal energy—human movement patterns are precisely classified to distinguish three common movement states: walking, running, and cycling.

[0186] 1. Walking: The final step frequency is 40-100 steps / minute, the micro-motion amplitude is threshold ≤ amplitude < 2 times the threshold, and the acoustic signal energy of the FB microphone is at a low level (the footstep vibration is smooth).

[0187] 2. Running: The final cadence is 100-200 steps / minute, the micro-motion amplitude is ≥2 times the threshold, and the acoustic signal energy of the FB microphone is at a high level (strong footstep vibration).

[0188] 3. Cycling: No stable periodic micro-motion cadence (although the filtered signal fluctuates, the peak time difference δT does not meet the periodic condition of "three or more consecutive stable cadences within 0.3s-1.5s"). The micro-motion amplitude is less than the threshold, but there are non-periodic low-frequency random micro-motions (after the road bumps are buffered by the soft tissue of the ear canal, they appear as irregular fluctuations in the 0.5-5Hz frequency band, which is different from the strong periodicity of gait). The FF microphone can detect the ambient airflow sound during cycling (50-200Hz continuous low-frequency energy). Combining the above non-periodic micro-motion and airflow sound characteristics, the cycling state is determined, and no steps are counted (there is no step count requirement for cycling, only state recognition).

[0189] Step 6: Output Results

[0190] The main control chip transmits the final determined motion state (walking / running / cycling / stationary), real-time cadence, cumulative steps, and other data to the mobile terminal (phone / watch) via Bluetooth communication protocol, and retains the detection data of the most recent hour locally on the earphone (not lost when power is off); if the user is detected to be in a "stationary state" for 5 minutes or more, the main control chip automatically reduces the sensor sampling frequency and enters a low-power detection mode to reduce the overall power consumption of the earphone.

[0191] (III) Adaptation schemes for different sensor types

[0192] This application supports two types of in-ear detection sensors: infrared optical sensors and capacitive sensors. Slight adaptations have been made to the signal processing stage to suit the characteristics of different sensors.

[0193] 1. Infrared optical sensor: The raw data is the ADC voltage value of the infrared receiver tube. When moving, the relative displacement between the earphone and the ear canal will cause changes in the amount of infrared light received, which will be manifested as periodic fluctuations in the voltage value. During signal processing, the focus is on eliminating weak interference from ambient light (adding a 50Hz power frequency filter), and the peak-to-peak threshold for micro-motion is set to 50mV.

[0194] 2. Capacitive sensor: The raw data is the capacitance-to-ADC value between the sensor and the ear canal. Small changes in the contact area during movement will cause periodic fluctuations in the capacitance. During signal processing, the focus is on eliminating electrostatic interference (by adding grounding filtering). The peak-to-peak threshold for micro-motion is set to 2pF.

[0195] (iv) Power consumption optimization design

[0196] This application, while ensuring detection accuracy, reduces the impact of motion detection on headphone battery life through multi-dimensional design:

[0197] 1. Dynamic sampling rate adjustment: When stationary, the sampling rate of the in-ear detection sensor is reduced to 50Hz and the microphone sampling rate is reduced to 8kHz; when in motion, the normal detection sampling rate is restored.

[0198] 2. Lightweight Algorithms: All signal processing and fusion decision algorithms are developed based on the embedded computing power of the main control chip, using fixed-point arithmetic instead of floating-point arithmetic to reduce the power consumption of algorithm execution;

[0199] 3. Triggered detection: When the headphones detect that they are not being worn, the motion detection process is immediately shut down, and data acquisition from the sensors and microphone is stopped.

[0200] (v) Adaptation to abnormal scenarios

[0201] 1. Loose earphone wearing scenario: If the data from the in-ear detection sensor shows a significant baseline drift (drift amount > 2 times the micro-motion threshold), it is determined that the earphone is loose. Step counting is paused and a "loose earphone wearing reminder" is sent to the mobile terminal via Bluetooth to avoid invalid detection;

[0202] 2. Sensor failure scenario: If a microphone / in-ear detection sensor continuously fails to output valid data, it will automatically switch to single sensor detection mode (if the microphone fails, only the in-ear detection signal will be used; if the in-ear sensor fails, only the acoustic signal will be used) to ensure that the motion detection function is not interrupted.

[0203] Corresponding to the above-described motion state detection method, this application also provides a motion state detection device. Figure 4 This is a schematic diagram of the structure of a motion state detection device provided in an embodiment of this application, as shown below. Figure 4 As shown, this embodiment provides a motion state detection device, which includes:

[0204] The receiving module 401 is used to receive the ambient sound signal collected by the first microphone, the acoustic vibration signal inside the ear canal collected by the second microphone, and the raw sampling signal collected by the in-ear detection sensor.

[0205] The first extraction module 402 is used to extract micro-motion features from the original sampled signal; wherein, the micro-motion features are used to reflect the features of minute contact changes between the earphone and the ear canal caused by human movement;

[0206] The second extraction module 403 extracts the acoustic step frequency from the vibration acoustic signal in the ear canal when the micro-motion signal meets the periodicity condition; wherein, the micro-motion signal is determined based on the micro-motion characteristics;

[0207] The first determining module 404 is used to determine the current wind noise intensity based on the ambient sound signal;

[0208] The second determining module 405 determines the motion state based on the current wind noise intensity, acoustic step frequency, and micro-motion characteristics.

[0209] In one possible implementation, the micro-motion features include micro-motion step frequency and micro-motion amplitude, and the first extraction module 402 is specifically used for:

[0210] The original sampled signal is deDC processed to obtain an intermediate signal with the DC component removed;

[0211] The intermediate signal is bandpass filtered to obtain the micro-motion signal;

[0212] The micro-motion step frequency and micro-motion amplitude are calculated based on the micro-motion signal.

[0213] In one possible implementation, the first extraction module 402 is further configured to:

[0214] When the micro-motion signal does not meet the periodicity condition:

[0215] The micro-motion amplitude is calculated based on the micro-motion signal; if the micro-motion amplitude is less than the preset amplitude threshold, and the ambient sound signal has continuous low-frequency airflow energy in the preset frequency band, the current motion state is determined to be cycling.

[0216] If the conditions for riding are not met, the state is determined to be stationary.

[0217] In one possible implementation, the second extraction module 403 is specifically used for:

[0218] When the micro-motion signal meets the periodicity condition, the vibration acoustic signal in the ear canal is bandpass filtered to match the bone conduction frequency range corresponding to the footstep sound.

[0219] The time interval for extracting signal energy peaks based on the filtered acoustic signal;

[0220] The acoustic step frequency is calculated based on the time interval.

[0221] In one possible implementation, the first determining module 404 is specifically used for:

[0222] The ambient sound signal is subjected to low-pass filtering in a preset wind noise analysis frequency band, the root mean square energy value of the filtered signal is calculated, and the current wind noise intensity is obtained by quantization based on the root mean square energy value.

[0223] Accordingly, the second determining module 405 is specifically used for:

[0224] Based on the current wind noise intensity, determine the first acceptance weight corresponding to the acoustic step frequency and the second acceptance weight corresponding to the micro-motion step frequency;

[0225] Based on the first and second acceptance weights, the acoustic step frequency and the micro-motion step frequency are weighted and fused to obtain the target step frequency;

[0226] The motion state is determined based on the target step frequency and micro-motion amplitude.

[0227] In one possible implementation, when determining the motion state based on the target step frequency and micro-motion amplitude, the second determining module 405 is specifically used for:

[0228] When the target step frequency is within the first preset step frequency range, the micro-motion amplitude is within the first preset amplitude range, and the energy of the filtered acoustic signal is less than the preset energy threshold, the current motion state is determined to be walking; the step frequency within the first preset step frequency range is greater than the preset amplitude threshold.

[0229] When the target cadence is within the second preset cadence range, the micro-motion amplitude is within the second preset amplitude range, and the energy of the filtered acoustic signal is greater than the preset energy threshold, the current movement state is determined to be running; wherein, the cadence within the second preset cadence range is not less than the cadence within the first preset cadence range, and the micro-motion amplitude within the second preset amplitude range is not less than the micro-motion amplitude within the first preset amplitude range.

[0230] In one possible implementation, the first extraction module 402 is further configured to: determine whether the micro-motion signal satisfies the periodicity condition;

[0231] Determining whether a micro-motion signal meets the periodicity condition includes:

[0232] Extracting the time difference between adjacent peaks from micro-motion signals;

[0233] Determine whether a preset number of consecutive time differences are within the first preset time difference range;

[0234] When a number of consecutive preset time differences are within the range of the first preset time difference, the micro-motion signal is determined to meet the periodicity condition.

[0235] The motion state detection device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0236] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5As shown, the electronic device 50 provided in this embodiment includes at least one processor 501 and a memory 502. Optionally, the device 50 further includes a communication component 503. The processor 501, memory 502, and communication component 503 are connected via a bus 504.

[0237] In a specific implementation, at least one processor 501 executes computer execution instructions stored in memory 502, causing at least one processor 501 to perform the above-described method.

[0238] The specific implementation process of processor 501 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0239] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0240] The memory may include high-speed memory (Random Access Memory, RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0241] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0242] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0243] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0244] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0245] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0246] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0247] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0248] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0249] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0250] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0251] Finally, it should be noted that other embodiments of this application will readily conceive of by those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and alterations may be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for detecting motion state, characterized in that, include: It receives ambient sound signals collected by the first microphone, acoustic vibration signals inside the ear canal collected by the second microphone, and raw sampling signals collected by the in-ear detection sensor; Micro-motion features are extracted from the original sampled signal; wherein, the micro-motion features are used to reflect the minute contact changes between the earphone and the ear canal caused by human movement; When the micro-motion signal meets the periodicity condition, the acoustic step frequency is extracted from the vibration acoustic signal in the ear canal; wherein, the micro-motion signal is determined based on the micro-motion characteristics; Determine the current wind noise intensity based on the ambient sound signals; The motion state is determined based on the current wind noise intensity, the acoustic step frequency, and the micro-motion characteristics.

2. The method according to claim 1, characterized in that, The micro-motion features include micro-motion step frequency and micro-motion amplitude. Extracting the micro-motion features from the original sampled signal includes: The original sampled signal is subjected to DC removal processing to obtain an intermediate signal with the DC component removed; The intermediate signal is bandpass filtered to obtain the micro-motion signal; The micro-motion step frequency and the micro-motion amplitude are calculated based on the micro-motion signal.

3. The method according to claim 2, characterized in that, The method further includes: When the micro-motion signal does not meet the periodicity condition: The micro-motion amplitude is calculated based on the micro-motion signal; if the micro-motion amplitude is less than a preset amplitude threshold, and the ambient sound signal has continuous low-frequency airflow energy in a preset frequency band, the current motion state is determined to be cycling. If the conditions for riding are not met, the state is determined to be stationary.

4. The method according to claim 2, characterized in that, Extracting the acoustic step frequency from the vibrational acoustic signal within the ear canal includes: The acoustic vibration signal within the ear canal is bandpass filtered to match the bone conduction frequency range corresponding to footsteps. The time interval for extracting signal energy peaks based on the filtered acoustic signal; The acoustic step frequency is calculated based on the time interval.

5. The method according to claim 4, characterized in that, Based on the ambient sound signal, determine the current wind noise intensity, including: The ambient sound signal is subjected to low-pass filtering of a preset wind noise analysis frequency band, the root mean square energy value of the filtered signal is calculated, and the current wind noise intensity is obtained by quantization based on the root mean square energy value. Accordingly, based on the current wind noise intensity, the acoustic step frequency, and the micro-motion characteristics, the motion state is determined, including: Based on the current wind noise intensity, determine the first acceptance weight corresponding to the acoustic step frequency and the second acceptance weight corresponding to the micro-motion step frequency; Based on the first acceptance weight and the second acceptance weight, the acoustic gait frequency and the micro-motion gait frequency are weighted and fused to obtain the target gait frequency; The motion state is determined based on the target step frequency and the micro-motion amplitude.

6. The method according to claim 5, characterized in that, The motion state is determined based on the target step frequency and the micro-motion amplitude, specifically including: When the target step frequency is within a first preset step frequency range, the micro-motion amplitude is within a first preset amplitude range, and the energy of the filtered acoustic signal is less than a preset energy threshold, the current movement state is determined to be walking; the amplitude within the first preset amplitude range is greater than the preset amplitude threshold. When the target cadence is within a second preset cadence range, the micro-motion amplitude is within a second preset amplitude range, and the energy of the filtered acoustic signal is greater than a preset energy threshold, the current movement state is determined to be running; wherein, the cadence in the second preset cadence range is not less than the cadence in the first preset cadence range, and the micro-motion amplitude in the second preset amplitude range is not less than the micro-motion amplitude in the first preset amplitude range.

7. The method according to any one of claims 2-6, characterized in that, The method further includes: determining whether the micro-motion signal meets the periodicity condition; Determining whether the micro-motion signal satisfies the periodicity condition includes: Extract the time difference between adjacent peaks from the micro-motion signal; Determine whether a preset number of consecutive time differences are within a first preset time difference range; When a predetermined number of consecutive time differences are within a first predetermined time difference range, the micro-motion signal is determined to satisfy the periodic condition.

8. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images