PDR step counting method assisted by sound

Abstract

The invention discloses a PDR step counting method assisted by sound, and belongs to the technical field of indoor navigation and localization. Triaxial acceleration and triaxial angular velocity data are collected through an inertial sensor of a smart phone, environmental audio data are collected through a microphone, and synchronization is carried out based on a unified time sequence; identifying the current carrying state of the mobile phone; starting a corresponding audio analysis model according to the carrying state, and extracting a specific acoustic event related to the gait cycle from the environment audio; performing preliminary gait detection on the inertial data, and extracting candidate gait cycles and step numbers; the sound event sequence and the inertial gait sequence are subjected to space-time matching and fusion, step counting correction is achieved through matching verification, missed step compensation and wrong step elimination, and the high-confidence-coefficient step number is output; interference of a single sensor in complex carrying modes and various motion states can be effectively suppressed, step counting accuracy, environmental adaptability and system robustness are remarkably improved, and the method is suitable for PDR positioning of large indoor scenes such as airports and terminals.

Description

Technical Field

[0001] This invention relates to the field of indoor navigation and positioning technology, and particularly to a method for improving the accuracy of pedestrian dead reckoning (PDR) step counting in large indoor environments such as airport terminals by fusing environmental sound characteristics with data from smartphone inertial sensors. This method is especially suitable for pedestrian step counting scenarios based on smartphone sensors. Background Technology

[0002] PDR technology is an autonomous positioning technology that does not rely on external positioning signals (such as GNSS and UWB). Its core principle is to continuously calculate the position of pedestrians by collecting inertial parameters (such as acceleration and angular velocity) during their movement and combining them with initial position information. The accuracy of step counting is the key prerequisite for determining the positioning accuracy of PDR.

[0003] Currently, mainstream PDR (Step Counting) methods primarily rely on inertial sensors such as accelerometers and gyroscopes built into smartphones. These sensors detect periodic acceleration or posture changes during pedestrian movement to identify gait cycles and thus count steps. However, this method, which solely relies on inertial sensors, has significant drawbacks. Firstly, smartphones are carried in various ways during pedestrian movement (e.g., handheld, held with arm swing, placed in a pocket), and these different carrying methods can cause varying degrees of interference to the inertial data collected by the sensors. For example, when held, the phone is easily affected by slight hand tremors; when placed in a pocket, it is easily affected by pressure from clothing. These interferences can distort the signals output by the inertial sensors, leading to missed or over-counted steps. Secondly, the diversity of pedestrian movement states (e.g., walking slowly, walking quickly, running, going up and down stairs) can also make the periodic characteristics of the inertial signals unstable, further reducing the accuracy of step counting.

[0004] To address the aforementioned issues, some optimization schemes have emerged in existing technologies. These include adaptive correction using inertial signal models for different carrying scenarios, or classification and recognition of gait features using machine learning algorithms. However, these schemes still rely solely on inertial sensor data, offering limited ability to suppress interference signals in complex scenarios, and their step counting error rates remain insufficient for high-precision positioning requirements. Therefore, there is an urgent need for a PDR step counting optimization method that can introduce new sensing dimensions and complement inertial sensor data to fundamentally improve step counting accuracy. Summary of the Invention

[0005] The technical problem to be solved by this invention is the inaccuracy of existing PDR (pedestrian chronograph) technology, which relies solely on inertial sensors such as accelerometers and gyroscopes. This invention provides a sound-assisted PDR method, which introduces a mobile phone audio acquisition module to obtain specific environmental sounds that are synchronized with the gait cycle when a pedestrian walks. Combined with inertial sensor data, it achieves dual verification and correction of the gait cycle, significantly improving the accuracy and environmental adaptability of step counting.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A sound-assisted PDR step counting method specifically includes the following steps;

[0008] Step S100, Initialization and Data Acquisition Startup: Use the inertial measurement unit of the smartphone to acquire inertial data, including three-axis acceleration data and three-axis angular velocity data, and use the microphone of the smartphone to acquire ambient audio data. Add a unified timestamp to the inertial data and ambient audio data to ensure the synchronization of the data stream.

[0009] Step S200, mobile phone carrying status recognition: Based on the inertial data collected in step S100, the current carrying status of the mobile phone is recognized in real time; the carrying status includes at least the hand-held status, the hand-held and arm-swinging status, and the pocket-placed status.

[0010] Step S300, gait-related acoustic event detection based on audio: Based on the mobile phone carrying status identified in step S200, the corresponding audio analysis model is activated to extract specific acoustic events related to the pedestrian's gait cycle from the environmental audio data;

[0011] Step S400, Preliminary gait detection based on inertial data: In parallel with step S300, the inertial data collected in step S100 is processed to perform preliminary gait detection and step counting, and an inertial gait sequence is obtained.

[0012] Step S500, Acoustic-Inertial Fusion and Step Count Correction: The periodic acoustic event sequence detected in step S300 and the inertial gait sequence initially detected in step S400 are spatiotemporally matched and fused. Step count correction is completed through matching verification, missing step compensation and erroneous step elimination, and the corrected high-confidence step count is output.

[0013] Step S600, Real-time update and output: After each gait cycle matching and correction is completed, the value of the step counter is updated in real time, and the current cumulative step count is output through the display module or communication module of the smartphone. At the same time, the corrected step count data and gait characteristics are stored locally to provide accurate stride length and gait information for subsequent PDR positioning calculation.

[0014] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, in step S100, the inertial data and audio data are synchronized using a microsecond-level monotonically increasing timestamp, and the sampling rate of the audio data is 44.1kHz.

[0015] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, in step S200, the mobile phone carrying status recognition is achieved through a machine learning classifier or rule-based logical judgment; specifically including:

[0016] Preprocessing and calibrating inertial sensor data;

[0017] Extract time-domain, frequency-domain, pose-domain, and cross-features from the preprocessed data;

[0018] The trained classification model is used to infer the extracted features and identify the current carrying state.

[0019] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, in step S200, the handheld state is: the mobile phone is in the hand with the screen facing the user, and the typical action is to operate the mobile phone or hold it.

[0020] Hand-held and arm-swinging state: The phone is held in the hand, and the user's arm swings naturally;

[0021] Pocket placement: The phone is placed in a pants or shirt pocket.

[0022] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, in step S300, the specific processing methods for different carrying states include:

[0023] When the hand is identified as being held, the pedestrian's hand is relatively stable, and the signal-to-noise ratio of the pedestrian's footsteps in the ambient noise is the highest. The spectral features of the sound are extracted using Mel-frequency cepstral coefficients (MFCC), and the footsteps signal is filtered out by combining the short-time energy threshold to remove environmental noise interference.

[0024] When the phone is identified as being held and swinging its arm, it will swing periodically with the arm, producing a wind sound that matches the frequency of the arm swing. Low-frequency noise is filtered out by a high-pass filter, and the time-domain peak and periodic features of the wind sound are extracted.

[0025] When the phone is identified as being placed in a pocket, it generates periodic friction with the clothing. The friction sound has a low frequency, so a low-pass filter is used to preserve the friction sound signal, and its periodic characteristics are extracted through autocorrelation analysis.

[0026] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, step S500 specifically includes the following steps;

[0027] Time series alignment: Aligning the time series of acoustic events and the time series of inertial gait events on a unified time axis;

[0028] Matching Verification: Set a period matching threshold δ. If the difference between the two periods is within the threshold range, the current sound period is determined to match the inertial gait period, confirming the validity of this step count and increasing the confidence of the gait period. If it is outside the threshold range, anomaly analysis is initiated.

[0029] Missed step compensation: If the inertial sensor does not detect a gait event, but there is a high-confidence acoustic event near the corresponding time point, it is determined that the inertial sensor missed the detection and one step is compensated.

[0030] False step removal: If an inertial sensor detects a gait event, but there are no related acoustic events within its corresponding time window, and the morphological characteristics of the inertial sensor event do not conform to typical gait, it is determined to be a false detection and removed from the total number of steps.

[0031] Confidence-weighted output: Assign a fused confidence score to each detected gait event, which is determined by the quality of the inertial sensor signal and the acoustic event signal; output the corrected number of high-confidence steps.

[0032] As a further preferred embodiment of the sound-assisted PDR step counting method of the present invention, the period matching threshold δ is dynamically adjusted according to the pedestrian's motion state and the stability of the sound period.

[0033] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0034] This invention provides a sound-assisted PDR step counting method. By introducing a mobile phone audio acquisition module to obtain specific environmental sounds that are synchronized with the gait cycle when a pedestrian walks, and combining them with inertial sensor data, the method achieves dual verification and correction of the gait cycle, significantly improving the accuracy and environmental adaptability of step counting.

[0035] 1) Multi-source complementarity enhances robustness: An audio perception dimension completely independent of the physical principle of the inertial sensor is introduced; when the inertial sensor signal is interfered with, the audio signal may still be clear; conversely, when the audio signal is interfered with in a noisy environment, the inertial sensor signal remains reliable; the two serve as backups for each other, significantly improving the robustness of the system in various complex scenarios.

[0036] 2) Strong scene adaptability: Different sound feature extraction strategies are designed for different carrying states of mobile phones. Combined with adaptive noise cancellation and online update mechanism of carrying state model, it can adapt to various carrying scenarios such as hand-held, arm swing, pocket and complex environmental sound interference, without the need for users to manually switch modes.

[0037] 3) Low power consumption and versatility: It only uses the inertial sensor and audio acquisition module built into the smartphone, without the need for additional hardware devices. It is low in cost and easy to promote and apply. It can be directly integrated into existing PDR positioning apps or navigation software.

[0038] 4) Provides a reliable foundation for PDR positioning: Accurate step count data is the core basis for step length estimation and position calculation in PDR positioning. The step count correction results of this invention can significantly improve the continuous accuracy of PDR positioning, especially suitable for indoor and underground passage scenarios where GPS signals are blocked. Attached Figure Description

[0039] Figure 1 This is an algorithm flowchart of a sound-assisted PDR step counting method according to the present invention;

[0040] Figure 2 This is a schematic diagram of acceleration data in various directions during pedestrian movement according to the present invention;

[0041] Figure 3 This is a schematic diagram illustrating the gait recognition effect of the inertial sensor multi-parameter constraint model of the present invention;

[0042] Figure 4 This is a schematic diagram of footstep sound features collected by the present invention using a handheld mobile phone.

[0043] Figure 5 This is a schematic diagram of the sound features collected by placing the mobile phone in a pocket according to the present invention. Detailed Implementation

[0044] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings:

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments. The purpose and effects of the present invention will become clearer. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention.

[0046] A sound-assisted PDR step counting method, such as Figure 1 As shown, the specific steps are as follows;

[0047] Step S100, Initialization and Data Acquisition Startup: The system uses the smartphone's inertial measurement unit to acquire triaxial acceleration and triaxial angular velocity data; simultaneously, it uses the smartphone's microphone to acquire ambient audio data. The system assigns a unified timestamp to the inertial and audio data to ensure data stream synchronization. The system initializes the pedometer counter, the phone's status indicator, and the characteristic sound sample library.

[0048] Step S200, Mobile Phone Carrying Status Identification: Based on the acceleration and angular velocity data collected in step S100, the current carrying status of the mobile phone is identified in real time through a machine learning classifier or rule-based logical judgment. The carrying status includes at least:

[0049] Handheld state: The phone is in the hand with the screen facing the user, and the typical action is operating the phone or holding it.

[0050] Hand-held and arm-swinging state: The phone is held in the hand, and the user's arm swings naturally;

[0051] Pocket placement: The phone is placed in a pants or shirt pocket.

[0052] Step S300, gait-related acoustic event detection based on audio: Based on the mobile phone carrying status identified in step S200, the corresponding audio analysis model is activated to extract specific acoustic events related to the pedestrian's gait cycle from the environmental audio data:

[0053] When identified as "handheld state", the pedestrian's hand is relatively stable, and the signal-to-noise ratio of the pedestrian's footsteps in the ambient sound is the highest. Mel frequency cepstral coefficients (MFCC) are used to extract the spectral features of the sound, and the footsteps signal is screened out in combination with the short-time energy threshold to remove environmental noise interference.

[0054] When the phone is identified as being in a "hand-held swinging state", the phone swings periodically with the arm, producing a wind sound that matches the swinging frequency. Low-frequency noise is filtered out by a high-pass filter (cutoff frequency 500Hz), and the time-domain peak and periodic features of the wind sound are extracted.

[0055] When the phone is identified as being in a pocket, it rubs periodically against the clothing. The friction sound has a low frequency, so a low-pass filter (cutoff frequency 1000Hz) is used to preserve the friction sound signal, and its periodic characteristics are extracted through autocorrelation analysis. A schematic diagram of the sound characteristics collected when the phone is in a pocket is shown below. Figure 5 As shown.

[0056] Step S400, Preliminary Gait Detection Based on Inertial Data: Parallel to step S300, the inertial data acquired in step S100 is processed to perform preliminary gait detection and step counting. Specifically, this includes filtering, denoising, and calculating the vector magnitude of the acceleration data, and then extracting candidate gait cycles and step counts through peak detection, zero-crossing detection, or waveform autocorrelation algorithms.

[0057] Step S500, Acoustic-Inertial Fusion and Step Count Correction: The periodic acoustic event sequence detected in step S300 and the inertial gait sequence initially detected in step S400 are spatiotemporally matched and fused.

[0058] Time series alignment: Aligning the time series of acoustic events and the time series of inertial gait events on a unified time axis;

[0059] Matching Verification: Set a period matching threshold δ. If the difference between the two periods is within the threshold range, the current sound period is determined to match the inertial gait period, confirming the validity of this step count and increasing the confidence of the gait period. If it is outside the threshold range, anomaly analysis is initiated.

[0060] Missed step compensation: If the inertial sensor does not detect a gait event, but there is a high-confidence acoustic event near the corresponding time point, it is determined that the inertial sensor missed the detection and one step is compensated.

[0061] False step removal: If an inertial sensor detects a gait event, but there are no related acoustic events within its corresponding time window, and the morphological characteristics of the inertial sensor event do not conform to typical gait, it is determined to be a false detection and removed from the total number of steps.

[0062] Confidence-weighted output: Each detected gait event is assigned a fused confidence score, which is jointly determined by the quality of the inertial sensor signal and the acoustic event signal. The final output is a corrected high-confidence step count.

[0063] Step S600, Real-time update and output: After each gait cycle matching and correction is completed, the value of the step counter is updated in real time, and the current cumulative step count is output through the display module or communication module of the smartphone. At the same time, the corrected step count data and gait characteristics are stored locally to provide accurate stride length and gait information for subsequent PDR positioning calculation.

[0064] Example:

[0065] Step 1: Initialization and Data Acquisition Startup

[0066] When a user walks inside the terminal (or other indoor area) with their mobile phone, the PDR application of this invention runs in the background. A schematic diagram of the pedestrian's acceleration data in various directions during movement is shown below. Figure 2As shown; a schematic diagram of the gait recognition effect of the inertial sensor multi-parameter constraint model of the present invention is shown below. Figure 3 As shown;

[0067] 1.1 Inertial Sensor Data Acquisition: Raw data from the three-axis accelerometer and three-axis gyroscope were synchronously read at a sampling frequency of 100Hz. To cope with possible strong electromagnetic interference within the terminal building, Kalman filtering was applied to the raw gyroscope data for real-time noise reduction.

[0068] 1.2 Audio Data Acquisition: The device's high-privilege audio API is invoked to acquire audio in mono mode with a sampling rate of 44.1kHz and a depth of 16 bits. To avoid audio stream interruptions due to system power-saving strategies, a persistent background audio session can be requested.

[0069] 1.3 High-precision synchronization: At startup, the system startup reference time T_base is recorded. For each inertial sensor data packet and audio data frame, a monotonically increasing timestamp t_imu and t_audio starting from T_base are appended to ensure strict time synchronization of cross-modal data.

[0070] Step 2: Mobile Phone Carrying Status Recognition

[0071] Mobile phone inertial sensors (mainly including accelerometers, gyroscopes, magnetometers, etc.) can capture the device's motion state, attitude changes, and environmental magnetic field information. Under different carrying states (such as holding while walking, making or receiving calls, swinging the arm while walking, and carrying in a pocket), the signal characteristics (such as acceleration amplitude, attitude stability, and frequency distribution) output by the sensors show significant differences. The core logic is to extract multi-dimensional features from the sensor signals. Common methods include feature-based machine learning, end-to-end deep learning-based methods, and pattern matching rule-based methods. This example describes feature-based machine learning.

[0072] 2.1 Data Collection and Labeling

[0073] 2.1.1 Sampling rate: accelerometer / gyroscope ≥ 50Hz (capturing fast movements such as swing arm and rotation), magnetometer ≥ 10Hz, and all sensor timestamps are synchronized (error ≤ 10ms).

[0074] 2.1.2 Sensor range settings: accelerometer ±16g (covering violent movements such as swing arm), gyroscope ±2000° / s (meeting the requirements for measuring rotational movements).

[0075] 2.2 Sensor Data Preprocessing

[0076] 2.2.1 Sensor Calibration

[0077] Accelerometer: Gravity removal (separating gravitational acceleration and motion acceleration through low-pass filtering) + zero bias calibration (collecting the average value of each axis when stationary and subtracting it as the zero bias value).

[0078] Gyroscope: Zero drift calibration (collect the average angular velocity of each axis when stationary, and subtract the zero drift value in real time).

[0079] Magnetometer: Hard iron calibration (eliminating interference from the device's own magnetic field) + soft iron calibration (eliminating external magnetic field distortion), which can be achieved through the ellipsoid fitting algorithm.

[0080] 2.2.2 Noise Filtering

[0081] A Butterworth low-pass filter (cutoff frequency 5Hz) is used to filter high-frequency noise (such as sensor jitter); the 3σ principle is used to remove extreme outliers (such as peak values ​​generated by sudden collisions).

[0082] 2.2.3 Data Synchronization and Resampling:

[0083] Accelerometer, gyroscope, and magnetometer data are aligned based on timestamps (e.g., magnetometer data is interpolated and resampled based on the accelerometer sampling time). A uniform data frame rate (e.g., 50Hz) is used to ensure consistency in feature extraction.

[0084] 2.3 Feature Extraction

[0085] From the preprocessed sensor data, four types of features are extracted: time domain, frequency domain, attitude domain, and statistical domain, covering the core differences in different carrying states.

[0086] 2.3.1 Time-domain characteristics

[0087] Statistical characteristics: mean, variance, standard deviation, maximum, minimum, peak-to-peak value (max-min), skewness, and kurtosis of each axis acceleration / gyroscope.

[0088] Fluctuation characteristics: accelerometer moving average (window size 0.5 seconds), sliding variance (reflects signal stability, such as extremely small sliding variance when making or receiving phone calls).

[0089] Peak characteristics: number of acceleration peaks (e.g., more peaks when the arm is swinging forward, fewer when making or receiving a phone call), and average peak interval.

[0090] 2.3.2 Frequency Domain Characteristics

[0091] Perform a Fast Fourier Transform (FFT) on the accelerometer / gyroscope data to extract: dominant frequency (the frequency with the highest energy, such as 1-2Hz when swinging the arm forward and 0.8-1.5Hz when walking vertically); dominant frequency energy percentage (such as ≥60% of the dominant frequency energy percentage on the horizontal axis when swinging the arm and no obvious dominant frequency when carried in a pocket); and spectral entropy (reflecting the uniformity of frequency distribution, with low spectral entropy for periodic movements and high spectral entropy for random postures).

[0092] 2.3.3 Attitude Domain Features

[0093] Attitude calculation: Complementary filtering (lightweight, real-time, suitable for mobile devices) or Kalman filtering (high accuracy, computationally intensive) is used to fuse accelerometer, gyroscope, and magnetometer data to output attitude angles (pitch angle θ, roll angle φ, yaw angle ψ).

[0094] Posture stability: Variance of posture angle (e.g., variance ≤ 5° when making or receiving a phone call, ≥ 20° when swinging the arm);

[0095] Number of attitude change events: The number of times the rate of change of attitude angle exceeds a threshold (e.g., 30° / s) per unit time (there is one change event when making or receiving a phone call, and multiple changes events when swinging the arm);

[0096] Gravity direction percentage: the angle between the accelerometer Z-axis and the gravity direction (e.g., the angle is ≤10° when making or receiving a phone call, and the angle is random when swinging the arm).

[0097] 2.3.4 Crossover Features

[0098] Acceleration-gyroscope correlation: For example, the Pearson correlation coefficient between the horizontal axis acceleration and angular velocity during swing arm operation is ≥0.7 (periodic synchronization).

[0099] Magnetometer signal strength: When carried in a pocket, the average amplitude of each axis of the magnetometer is 10%-20% lower than when held in hand (due to clothing obstruction).

[0100] Time characteristics: Duration of state (e.g., answering a phone call usually lasts ≥10 seconds, arm swinging forward lasts longer, and temporary hand holding may only last a few seconds).

[0101] 2.4 Model Training and Classification

[0102] 2.4.1 Feature Preprocessing

[0103] Feature normalization: Standardize all features to the [0, 1] interval (e.g., Min-Max normalization) to eliminate the influence of dimensions.

[0104] Feature selection: Use ANOVA or random forest feature importance to remove redundant features (such as some highly correlated statistical features) to reduce model complexity.

[0105] 2.4.2 Model Selection and Training

[0106] From simple to complex, you can choose to train models such as logistic regression, support vector machine, random forest, lightweight neural network, gradient boosting tree, etc.

[0107] 2.4.3 Model Training Process

[0108] The training, validation, and test sets are divided in a 7:2:1 ratio (ensuring each set contains all carried states); traditional machine learning (LR / SVM / RF) uses the sklearn library for fast training and optimizes hyperparameters (such as the C value of SVM and the number of trees in RF) through grid search; lightweight neural networks use TensorFlow / Keras to build 2-3 layer MLPs (input layer = number of features, hidden layer = 64 / 32, output layer = number of states), with ReLU as the activation function, Adam as the optimizer, and cross-entropy as the loss function.

[0109] Model evaluation uses accuracy, precision, recall, and F1 score to assess model performance.

[0110] Perform quantization (INT8 quantization) and pruning on the neural network model to reduce computational load and memory usage (for mobile devices).

[0111] 2.5 Real-time recognition and post-processing

[0112] 2.5.1 Real-time Inference

[0113] Sliding window: A 2-second window (containing 100 sampling points, 50Hz sampling rate) is used, and the window slides once every 0.5 seconds to extract features within the window.

[0114] Model inference: Input the features into the compressed model and output the probability of each state.

[0115] State decision: Select the state with the highest probability as the current recognition result (probability threshold ≥ 0.7, otherwise output "unknown state").

[0116] 2.5.2 Post-processing optimization

[0117] Time smoothing: The state is confirmed only after three consecutive windows output the same state (to avoid momentary false detections).

[0118] State transition constraints: Set a reasonable state transition matrix (e.g., "making a phone call" will not directly transition to "swinging the arm forward", but must first go through "hand-held forward") to filter out impossible state transitions.

[0119] Dynamic threshold adjustment: The probability threshold is dynamically adjusted according to the environment (e.g., the threshold is increased to 0.8 in a noisy environment and decreased to 0.6 in a quiet environment).

[0120] Step 3: Audio-based gait-related acoustic event detection

[0121] Based on the results of the phone carrying status recognition, the filtering frequency for environmental noise is selected. For example, let's take the case of collecting footsteps while holding the phone normally.

[0122] 3.1 Analysis of Key Acoustic Features of Footsteps

[0123] Footsteps are essentially sound wave signals resulting from the impact vibration of the shoe sole against the ground and the shift of the body's center of gravity. Their waveforms are highly distinctive. A single step exhibits a "short peak" and "rapid decay" pattern (impact phase 0.1-0.3 seconds, decay phase 0.2-0.5 seconds), with peak amplitudes significantly higher than ambient noise (signal-to-noise ratio ≥15dB in quiet environments). In the frequency domain, the main energy is concentrated in the 200-2000Hz range (mid-to-low frequency band), differing from speech (1000-4000Hz) and ambient noise (broadband or single frequency). In terms of timing, the stride frequency during normal walking is 0.8-1.5 steps / second (i.e., step interval 0.67-1.25 seconds), while the stride frequency during running is 1.5-3 steps / second, exhibiting a stable temporal periodicity.

[0124] 3.2 Footstep Sound Signal Preprocessing

[0125] 3.2.1 Pre-emphasis: The mid-to-high frequency components are boosted by a high-pass filter (cutoff frequency 100Hz) to compensate for the air attenuation of footsteps.

[0126] 3.2.2 Framing and Windowing: Continuous audio is divided into short frames (frame length 20-30ms, frame shift 50%), and a Hanning window is added to each frame (to reduce inter-frame spectral leakage).

[0127] 3.2.3 Noise Suppression: Static noise (such as background air conditioning noise) is suppressed by spectral subtraction, which estimates the noise spectrum through the silent segment and subtracts it from the signal; dynamic noise (such as sudden coughing) is suppressed by adaptive thresholding, which adjusts the noise threshold according to the local signal energy.

[0128] 3.2.4 Normalization: Normalize the signal amplitude to the range of [-1, 1] to eliminate the influence of different recording volumes.

[0129] 3.3 Footstep Sound Feature Detection

[0130] A schematic diagram of footstep sound characteristics captured by handheld mobile phone is shown below. Figure 4 As shown.

[0131] Depending on the environmental characteristics, footstep detection can be achieved using methods such as time-domain peak detection, model matching, frequency-domain assisted methods, and machine learning classification.

[0132] Method A: Time-domain peak detection (suitable for low-noise scenarios, simple and efficient)

[0133] (1) Peak filtering: Traverse the preprocessed signal, detect local maximum values ​​(peaks), set an amplitude threshold (such as 3 times higher than the global signal average), and filter out candidate peaks.

[0134] (2) Peak Verification

[0135] 1) Amplitude verification: The candidate peak value must be higher than the noise threshold (obtained through statistics of silent segments) and lower than 80% of the maximum signal value (to avoid peak values ​​with saturation distortion).

[0136] 2) Width verification: The duration of the "rising edge + falling edge" before and after the peak should be within 0.1-0.8 seconds (which is consistent with the impact and decay cycle of footsteps).

[0137] (3) Interval filtering: The time interval between adjacent candidate peaks should be 0.3-1.5 seconds (walking) or 0.2-0.7 seconds (running) to exclude false detections of echoes and double peaks (such as two impacts on the sole of the shoe).

[0138] Method B: Model matching method (suitable for complex noisy scenarios, with stronger robustness)

[0139] (1) Template construction: Extract the waveform periodic features of typical footsteps (such as "rise-peak-fall-zero crossing") from the labeled samples and use them as footstep feature models.

[0140] (2) Sliding matching: Cross-correlation is performed between the model and the preprocessed signal, and positions with cross-correlation coefficients higher than a threshold (e.g., 0.7) are identified as candidate events.

[0141] (3) Model update: Adaptive templates are used (the template is updated every 10 steps detected) to adapt to changes in footsteps in different scenarios (such as the difference in footsteps from cement floor to carpet).

[0142] Method C: Frequency domain feature-assisted detection (improving robustness in complex scenes)

[0143] When the false positive rate of time-domain detection is high, further screening is performed by combining frequency-domain features:

[0144] (1) Perform FFT on each frame of signal to calculate the spectral energy distribution.

[0145] (2) Extract frequency domain features: such as the energy proportion of the 200-2000Hz frequency band (the energy proportion of footsteps in this frequency band is ≥60%) and the center of the spectrum (the center of the spectrum of footsteps is concentrated in 500-1000Hz).

[0146] (3) Only candidate events that simultaneously satisfy the time-domain peak characteristics and frequency-domain energy characteristics are retained.

[0147] Method D: Machine Learning Classification

[0148] (1) Divide the audio into short frames (e.g., 50ms).

[0149] (2) Extract the features such as Mel frequency cepstral coefficients (MFCC), short time energy (STE), and zero crossing rate of each frame.

[0150] (3) Use a pre-trained binary classifier (such as SVM or CNN) to determine whether the frame contains footsteps.

[0151] 3.4 Event Deduplication and Counting (Eliminating False Detections and Ensuring Counting Accuracy)

[0152] 3.4.1 Deduplication Logic: The same footstep sound may generate multiple peaks (such as heel + forefoot landing), which need to be merged by time interval. If the interval between adjacent candidate events is <0.2 seconds, it is determined to be the same footstep, and the peak with the largest amplitude is retained.

[0153] 3.4.2 Anomaly Filtering: If the interval between consecutive candidate events is <0.3 seconds (ultra-fast cadence) or >2 seconds (long pause), it is marked as an anomaly and not included in the count.

[0154] 3.4.3 Cumulative Count: The final number of steps is obtained by summing up all the candidate events that have passed the verification.

[0155] Step 4: Preliminary Gait Detection Based on Inertial Data PDR

[0156] Common gait calculation methods include the zero-point crossing method, peak acceleration detection method, and multi-parameter constrained gait detection method. This example illustrates the multi-parameter constrained gait detection method.

[0157] 4.1 Data Preprocessing: The influence of gravitational acceleration is removed from the Euler angle vector data, and the acceleration data is filtered using mean filtering.

[0158] 4.2 Peak Detection: First, obtain the maximum extreme point of acceleration based on the trend of acceleration change, and then compare the amplitude with the empirical threshold constraint parameter th. pk The comparison is performed, and when the amplitude is greater than the threshold, the point is recorded as a peak, and the quantity constraint parameter num is increased. pk If a peak reappears without experiencing a zero point and a trough, the amplitudes of the two peaks are compared. The peak with the larger amplitude is set as the peak, and the peak with the smaller amplitude is a pseudo-peak. The peak detection model is shown in equations (1) and (2).

[0159] 4.3 Zero Point Detection: After detecting a zero point, record the zero point position and increment the quantity constraint parameter num. zero .

[0160] 4.4 Valley Detection: The logic process is similar to peak detection, utilizing the constraints of peak-valley correspondence and peak-valley-zero correspondence to remove false valleys. This process maintains num. pk ,num vy ,num zero The values ​​of the three quantity constraint parameters are increased synchronously, and the peak detection model is shown in equations (3) and (4).

[0161] 4.5 Verification: Compare the amplitude difference between the marked peaks and troughs within the period with the empirical threshold value Δacc for the peak and trough amplitudes. pk-vy Compare and judge the correctness of the test.

[0162]

[0163] Step 5: Sound consistency and step count correction

[0164] 5.1 Dynamic setting of matching threshold δ

[0165] The matching threshold δ is not a fixed value, but is dynamically adjusted according to the motion state and the stability of the sound period to ensure matching accuracy in different scenarios: when moving slowly and the sound period stability σ_s ≤ 0.1s (stable), the matching threshold δ = 0.3s; when moving slowly and the sound period stability σ_s > 0.1s (unstable), the matching threshold δ = 0.2s; when walking at a constant speed and the sound period stability σ_s ≤ 0.1s (stable), the matching threshold δ = 0.2s; when walking at a constant speed and the sound period stability σ_s > 0.1s (unstable), the matching threshold δ = 0.15s.

[0166] 5.2 Time Alignment and Period Matching

[0167] Time window alignment: Based on the start time of the sound cycle T_s, a time window of ±0.5s is extracted to determine whether there is a corresponding inertial peak (i.e. the start or end point of the inertial gait cycle) within the window, ensuring that the two cycles are aligned in the time dimension (avoiding cycle misalignment caused by sensor start-up delay).

[0168] Periodic numerical matching: For the aligned inertial gait period T_ij and sound period T_s, calculate the absolute value of the difference |T_ij-T_s|:

[0169] If |T_ij-T_s|≤δ and the sound period is valid, then the step count corresponding to the inertial peak is determined to be "valid" and the step count is retained.

[0170] If |T_ij-T_s|>δ, or the sound period is invalid, then mark the inertial peak as "suspicious step count" and proceed to the anomaly analysis stage.

[0171] 5.3 Anomaly Analysis and Step Count Correction

[0172] 5.3.1 False Detection Removal

[0173] The following situations are considered inertial false detections, and the corresponding suspicious step counts are removed:

[0174] Case 1: There are multiple suspicious peaks in the inertial gait periodic sequence, but the sound period T_s is stable and there is no corresponding sound period matching. In this case, it is mostly false peaks caused by phone shaking or collision of items in the bag.

[0175] Scenario 2: The amplitude of a single inertial peak is more than 20% lower than the corresponding dynamic threshold and there is no corresponding sound period. In this case, it is mostly a false trigger caused by sensor noise.

[0176] Scenario 3: At the time point corresponding to the inertial peak, the sound signal shows no significant energy change (short-term energy < 50% of average energy), which is mostly due to inertial changes caused by non-motion behaviors (such as raising your hand to check your phone).

[0177] 5.3.2 Supplementing Missed Detections

[0178] The following situations are considered as inertial missed detections, and the steps should be supplemented based on the sound cycle:

[0179] Case 1: The sound period T_s is stable (σ_s≤0.1s), and two or more identical T_s occur consecutively (e.g., T_s=0.7s, three consecutive periods), but there is no corresponding peak in the inertial gait period sequence. In this case, it is mostly due to missed detection caused by the attenuation of the inertial signal. The steps are supplemented according to the number of sound periods.

[0180] Case 2: If there is a blank interval with an interval greater than 1.5*T_s in the inertial gait periodic sequence, and the sound signal energy is stable in the interval, it is determined to be an inertial missed detection, and one additional step count is added.

[0181] 5.4 Step Count Results Fusion Output

[0182] 5.4.1 Cumulative Valid Steps: Sum the "Matching Valid Steps" and "Supplementary Steps" to obtain the cumulative step count within the current time window;

[0183] 5.4.2 Result Smoothing: A moving average is used to smooth the cumulative step count to avoid fluctuations in step count caused by abnormal single sound cycles;

[0184] 5.4.3 Data Synchronization and Storage: The corrected step count, sound period T_s, inertial gait period sequence, carrying status, and motion status tags are synchronously stored on the mobile phone to provide data support for subsequent PDR positioning step length estimation.

[0185] Step 6: Real-time updates and output

[0186] After each gait cycle correction is completed, the step counter value is updated in real time, and the cumulative step count is output through the mobile phone display module. At the same time, the corrected step count data, gait cycle characteristics and current carrying status are stored locally on the mobile phone, providing accurate data support for subsequent PDR positioning in the terminal to estimate stride length and calculate location.

[0187] This invention provides a sound-assisted PDR step counting method. By introducing a mobile phone audio acquisition module to obtain specific environmental sounds that are synchronized with the gait cycle when a pedestrian walks, and combining them with inertial sensor data, the method achieves dual verification and correction of the gait cycle, significantly improving the accuracy and environmental adaptability of step counting.

[0188] This invention introduces an audio perception dimension that is completely independent of the physical principles of inertial sensors; when the inertial sensor signal is interfered with, the audio signal may still be clear; conversely, when the audio signal is interfered with in a noisy environment, the inertial sensor signal remains reliable; the two serve as backups for each other, significantly improving the robustness of the system in various complex scenarios.

[0189] This invention designs differentiated sound feature extraction strategies for different carrying states of mobile phones, and combines adaptive noise cancellation and online update mechanism of carrying state model, which can adapt to various carrying scenarios such as hand-held, arm-swinging, pocket and complex environmental noise interference, without requiring users to manually switch modes.

[0190] This invention utilizes only the inertial sensor and audio acquisition module built into a smartphone, without the need for additional hardware. It is low-cost, easy to promote and apply, and can be directly integrated into existing PDR positioning apps or navigation software.

[0191] The accurate step count data of this invention is the core basis for step length estimation and position calculation in PDR positioning. The step count correction results of this invention can significantly improve the continuous accuracy of PDR positioning, and are especially suitable for indoor and underground passage scenarios where GPS signals are blocked.

[0192] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0193] It will be understood by those skilled in the art that the above descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention. All technical features in this embodiment can be freely combined according to actual needs.

[0194] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A sound-assisted PDR (pedometer detection) method, characterized in that: Specifically, it includes the following steps; Step S100, Initialization and Data Acquisition Startup: Use the inertial measurement unit of the smartphone to acquire inertial data, including three-axis acceleration data and three-axis angular velocity data, and use the microphone of the smartphone to acquire ambient audio data. Add a unified timestamp to the inertial data and ambient audio data to ensure the synchronization of the data stream. Step S200, mobile phone carrying status recognition: Based on the inertial data collected in step S100, the current carrying status of the mobile phone is recognized in real time; the carrying status includes at least the hand-held status, the hand-held and arm-swinging status, and the pocket-placed status. Step S300, gait-related acoustic event detection based on audio: Based on the mobile phone carrying status identified in step S200, the corresponding audio analysis model is activated to extract specific acoustic events related to the pedestrian's gait cycle from the environmental audio data; Step S400, Preliminary gait detection based on inertial data: In parallel with step S300, the inertial data collected in step S100 is processed to perform preliminary gait detection and step counting, and an inertial gait sequence is obtained. Step S500, Acoustic-Inertial Fusion and Step Count Correction: The periodic acoustic event sequence detected in step S300 and the inertial gait sequence initially detected in step S400 are spatiotemporally matched and fused. Step count correction is completed through matching verification, missing step compensation and erroneous step elimination, and the corrected high-confidence step count is output. Step S600, Real-time update and output: After each gait cycle matching and correction is completed, the value of the step counter is updated in real time, and the current cumulative step count is output through the display module or communication module of the smartphone. At the same time, the corrected step count data and gait characteristics are stored locally to provide accurate stride length and gait information for subsequent PDR positioning calculation.

2. The sound-assisted PDR step counting method according to claim 1, characterized in that: In step S100, the inertial data and audio data are synchronized using a microsecond-level monotonically increasing timestamp, and the audio data sampling rate is 44.1kHz.

3. The sound-assisted PDR step counting method according to claim 1, characterized in that: In step S200, the mobile phone carrying status recognition is achieved through a machine learning classifier or rule-based logical judgment; specifically, it includes: Preprocessing and calibrating inertial sensor data; Extract time-domain, frequency-domain, pose-domain, and cross-features from the preprocessed data; The trained classification model is used to infer the extracted features and identify the current carrying state.

4. The sound-assisted PDR step counting method according to claim 1, characterized in that: In step S200, the handheld state is as follows: the mobile phone is in the hand with the screen facing the user, and the typical action is to operate the mobile phone or hold it. Hand-held and arm-swinging state: The phone is held in the hand, and the user's arm swings naturally; Pocket placement: The phone is placed in a pants or shirt pocket.

5. The sound-assisted PDR step counting method according to claim 1, characterized in that: In step S300, the specific processing methods for different carrying states include: When the hand is identified as being held, the pedestrian's hand is relatively stable, and the signal-to-noise ratio of the pedestrian's footsteps in the ambient noise is the highest. The spectral features of the sound are extracted using Mel-frequency cepstral coefficients (MFCC), and the footsteps signal is filtered out by combining the short-time energy threshold to remove environmental noise interference. When the phone is identified as being held and swinging its arm, it will swing periodically with the arm, producing a wind sound that matches the frequency of the arm swing. Low-frequency noise is filtered out by a high-pass filter, and the time-domain peak and periodic features of the wind sound are extracted. When the phone is identified as being placed in a pocket, it rubs periodically against the clothing. The friction sound has a low frequency, so a low-pass filter is used to preserve the friction sound signal, and its periodic characteristics are extracted through autocorrelation analysis.

6. The sound-assisted PDR step counting method according to claim 1, characterized in that: Step S500 specifically includes the following steps; Time series alignment: Aligning the time series of acoustic events and the time series of inertial gait events on a unified time axis; Matching verification: Set a period matching threshold δ. If the difference between the two periods is only within the threshold range, it is determined that the current sound period matches the inertial gait period, confirming that the step count is valid and enhancing the confidence of the gait period. If the value is outside the threshold range, then anomaly analysis will be initiated: Missed step compensation: If the inertial sensor does not detect a gait event, but there is a high-confidence acoustic event near the corresponding time point, it is determined that the inertial sensor missed the detection and one step is compensated. False step removal: If an inertial sensor detects a gait event, but there are no related acoustic events within its corresponding time window, and the morphological characteristics of the inertial sensor event do not conform to typical gait, it is determined to be a false detection and removed from the total number of steps. Confidence-weighted output: Assign a fused confidence score to each detected gait event, which is determined by the quality of the inertial sensor signal and the acoustic event signal; output the corrected number of high-confidence steps.

7. A sound-assisted PDR step counting method according to claim 6, characterized in that: The period matching threshold δ is dynamically adjusted based on the pedestrian's movement state and the stability of the sound period.

Images