A method and system for identifying bipedal foot contact states based on monocular video

Abstract

This invention provides a method and system for identifying bipedal foot-to-ground contact states based on monocular video, relating to the field of data processing technology. The method includes: acquiring a two-dimensional keypoint sequence of human posture from monocular video; performing robust outlier suppression and smoothing differentiation on the two-dimensional keypoint sequence to obtain a smoothed keypoint coordinate and velocity sequence; extracting multiple kinematic features related to the bipedal foot-to-ground contact state; determining the estimated direction of human movement through a viewpoint adaptation mechanism, and projecting the foot velocity based on the direction of human movement to obtain viewpoint-adaptive velocity features; modeling the bipedal foot-to-ground contact state using an unsupervised two-component Gaussian mixture model and calculating the posterior probability that the feet are in the support phase; performing joint temporal decoding of the bipedal foot-to-ground contact state using a four-state coupled hidden Markov model and a Viterbi decoding algorithm to obtain a temporally consistent bipedal foot-to-ground contact state sequence; and completing the identification of the bipedal foot-to-ground contact state by extracting ground contact events and ground lift events.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for identifying bipedal foot contact states based on monocular video. Background Technology

[0002] In fields such as sports biomechanics, rehabilitation medicine, and sports science, accurately identifying the contact state between the feet and the ground (i.e., foot-ground contact state) and key gait events (such as heel-strike, HS; toe-off, TO) during walking or running is the foundation for gait cycle division, spatiotemporal parameter calculation (such as stride frequency, stride length, and support phase ratio), and subsequent dynamic analysis (such as ground reaction force estimation).

[0003] Currently, a typical monocular video foot-ground contact state recognition method obtains the two-dimensional trajectory of key points such as the ankle joint in the video through a human pose estimation algorithm. Then, based on the vertical height or velocity of the ankle joint, a fixed threshold is set, and combined with hysteresis rules, to determine whether the foot is in contact with the ground. This method is simple to implement and requires no additional hardware.

[0004] However, existing technologies rely on fixed thresholds that cannot adapt to different shooting angles, individual differences, and walking speeds. When the camera is not parallel to the direction of movement (such as an oblique angle), the motion features in the image coordinate system will be distorted, causing the judgment based on the fixed-direction threshold to completely fail, resulting in severely distorted recognition results. At the same time, the threshold needs to be manually adjusted repeatedly for different scenarios, lacking generalization ability and practicality in real-world, ever-changing environments. Summary of the Invention

[0005] To address the limitations of existing technologies that rely on fixed thresholds, which cannot adapt to different shooting angles, individual differences, and walking speeds, and to the distortion of motion features in the image coordinate system when the camera is not parallel to the direction of movement (e.g., at an oblique angle), leading to the complete failure of judgments based on fixed-direction thresholds and severely distorted recognition results, this invention provides a method and system for identifying bipedal foot contact states based on monocular video. Furthermore, the thresholds require repeated manual adjustments for different scenarios, lacking generalization ability and practicality in real-world, variable environments.

[0006] The technical solutions provided by the embodiments of the present invention are as follows: A first aspect of this invention provides a method for identifying bipedal foot-to-ground contact state based on monocular video, comprising: S1: Obtain the 2D keypoint sequence of human pose in monocular video; S2: Robust outlier suppression and smoothing derivative are performed on the two-dimensional keypoint sequence to obtain a smoothed keypoint coordinate and velocity sequence; S3: Based on smoothed key point coordinates and velocity sequences, extract multiple kinematic features related to the bipedal ground contact state; S4: Based on multiple kinematic features, the estimated direction of human movement is determined through a perspective adaptive mechanism, and the foot velocity is projected onto the direction of human movement to obtain perspective adaptive velocity features. S5: Based on multiple kinematic features and perspective-adaptive velocity features, an unsupervised two-component Gaussian mixture model is used to model the state of both feet touching the ground and calculate the posterior probability that both feet are in the support phase. S6: Based on the posterior probability, the two-foot-touching state is jointly temporally decoded using a four-state coupled hidden Markov model and the Viterbi decoding algorithm to obtain a temporally consistent two-foot-touching state sequence. S7: Based on the bipedal ground contact state sequence, the bipedal ground contact state is identified by extracting ground contact events and ground lift events.

[0007] A second aspect of the present invention provides a bipedal foot-ground contact state recognition system based on monocular video, comprising: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the bipedal foot contact state recognition method based on monocular video as described in the first aspect.

[0008] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the bipedal foot contact state recognition method based on monocular video as described in the first aspect.

[0009] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this invention, by using a perspective adaptation mechanism and single-sequence unsupervised probabilistic modeling, the problem of recognition failure caused by the inability of fixed thresholds to adapt to different shooting perspectives, individual differences and walking speeds in the prior art is effectively solved. It can automatically correct motion features under complex shooting conditions such as oblique perspectives, and adaptively learn the state boundary for each input sequence without manual parameter tuning or reliance on labeled data. This significantly improves the generalization ability, robustness and practicality of monocular video foot contact state recognition method in different real-world scenarios. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating a method for identifying bipedal foot-to-ground contact state based on monocular video, provided in an embodiment of the present invention.

[0012] Figure 2 This invention provides a four-state coupled HMM state diagram and a schematic diagram of the main transition relationships.

[0013] Figure 3 This is a schematic diagram of the physical quantities corresponding to the four elements provided in an embodiment of the present invention.

[0014] Figure 4 This is a schematic diagram of the structure of a bipedal foot contact state recognition system based on monocular video, provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0016] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0017] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0018] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0019] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0020] Reference manual attached Figure 1 The diagram shows a flowchart of a method for identifying bipedal foot contact state based on monocular video, provided by an embodiment of the present invention.

[0021] This invention provides a method for identifying bipedal foot-to-ground contact state based on monocular video. This method can be implemented using a monocular video-based bipedal foot-to-ground contact state identification device, which can be a terminal or a server. The processing flow of the monocular video-based bipedal foot-to-ground contact state identification method may include the following steps: S1: Obtain the 2D keypoint sequence of human pose from monocular video.

[0022] Among them, monocular video human pose refers to the position and pose of the main joints of the human body on a two-dimensional image plane estimated by computer vision algorithms from a video captured by a single camera.

[0023] Among them, the two-dimensional keypoint sequence refers to a series of two-dimensional coordinates of human keypoints arranged in chronological order, output by the human pose estimation network.

[0024] In one possible implementation, sub-steps S1A and S1B are included after S1 and before S2: S1A: Obtain the temporal information corresponding to the 2D keypoint sequence. The temporal information includes the frame index, timestamp, and scale factor.

[0025] The frame index refers to the sequential number of the video frame, which usually starts from 0 or 1 and increases by integers.

[0026] The timestamp refers to the specific time point corresponding to each frame of the video.

[0027] The scale factor refers to the conversion ratio from image pixel units to actual physical units.

[0028] S1B: Based on time information, converts the pixel coordinates of a two-dimensional keypoint sequence into metric coordinates.

[0029] Among them, the pixel coordinate sequence refers to the original two-dimensional keypoint sequence in pixels.

[0030] The metric coordinate sequence refers to the keypoint coordinate sequence in meters obtained by multiplying the pixel coordinate sequence by a scale factor.

[0031] Specifically, the input is a two-dimensional keypoint sequence `keypoints_xyc` obtained from a monocular video through a human pose estimation network, with the shape being... The third dimension is Simultaneously input the frame index `frame_idx`, timestamp `timestamp_s`, and scale factor `s` (m / px). Coordinate system convention: the image coordinate x-axis is positive to the right, and the y-axis is positive downwards. Convert pixel coordinates to metric coordinates uniformly. .

[0032] in, This represents the x-axis coordinate of keypoint j in the metric coordinate system of frame i. This represents the y-coordinate of keypoint j in the metric coordinate system of frame i. Indicates the scale factor. This represents the x-axis coordinate of key point j in the i-th frame in the pixel coordinate system. This represents the y-axis coordinate of key point j in the i-th frame in the pixel coordinate system.

[0033] when If the coordinates are not finite, the point is considered missing (denoted as NaN) to avoid noise propagating into subsequent velocity / angular velocity calculations.

[0034] in, Let (0-1) represent the confidence level of key point j in the i-th frame. The key point confidence threshold is represented by conf_thr.

[0035] It should be noted that those skilled in the art can set the confidence threshold according to actual needs, and this invention does not limit it.

[0036] In this embodiment of the invention, by unifying the coordinate system and physical scale, standard input data with real physical meaning and comparability is provided for subsequent kinematic analysis, laying the foundation for the cross-scenario application of the method.

[0037] S2: Robust outlier suppression and smoothing derivative are applied to the two-dimensional keypoint sequence to obtain a smoothed keypoint coordinate and velocity sequence.

[0038] Robust outlier suppression refers to a method specifically designed to identify and process erroneous or noisy data points in time-series data that deviate significantly from the normal pattern.

[0039] Among them, smoothing derivative refers to the process of calculating the derivative (speed) of the data after smoothing the data to suppress high-frequency noise.

[0040] In one possible implementation, S2 specifically includes sub-steps S201 to S204: S201: Calculate the median and MAD of the two-dimensional keypoint sequence within the sliding window using the Hampel robust filtering algorithm.

[0041] Among them, the Hampel robust filtering algorithm refers to an outlier detection and replacement algorithm based on sliding window and robust statistics.

[0042] In this context, a sliding window refers to a local subset of data formed by selecting data points within a certain range before and after each data point in the sequence when processing time-series data.

[0043] MAD refers to the median absolute deviation.

[0044] S202: Based on the median and MAD, replace outliers in the two-dimensional keypoint sequence to obtain a robust coordinate sequence.

[0045] Specifically, to suppress velocity amplification noise caused by key point jitter, this invention performs two-level processing on the key joint trajectory: Hampel robust outlier suppression and Savitzky-Golay (SG) smoothing derivative.

[0046] Optionally, Hampel filtering specifically includes applying the filter to a one-dimensional sequence. In length Calculate the median and MAD within a sliding window, and replace outliers: .

[0047] .

[0048] .

[0049] in, This represents the median. This represents median operations. This represents the k-th data point within the sliding window. This indicates the half-width of the window. , Indicates the length of the sliding window. Indicates the absolute deviation of the median. This represents the i-th data point in the one-dimensional keypoint coordinate sequence to be processed. This represents a conditional statement. This represents the threshold coefficient (preferably 3), window length, and sampling interval. The relationship is: .

[0050] in, Indicates taking an odd function, This represents the rounding function. This indicates the preset Hampel filter time window length. Indicates the sampling time interval.

[0051] It should be noted that those skilled in the art can set the value of the threshold coefficient according to actual needs, and this invention does not limit it.

[0052] S203: The robust coordinate sequence is smoothed using the SG smoothing algorithm to obtain the smoothed key point coordinates.

[0053] Among them, the SG smoothing algorithm (Savitzky-Golay smoothing algorithm) refers to a convolution smoothing method based on local polynomial least squares fitting.

[0054] S204: Take the first derivative of the coordinates of the smoothed key points to obtain the velocity sequence.

[0055] Specifically, SG smoothing and the first derivative are obtained by performing SG smoothing on the coordinate sequence after Hampel processing. and in Taking the derivative gives the velocity: .

[0056] SG window length: .

[0057] If key points are missing for a short period of time, linear interpolation can be performed on the missing segments to ensure stability of the derivative.

[0058] in, Represents a sequence The smoothed coordinate sequence obtained after SG smoothing is the value of the i-th frame. This represents the Savitzky-Golay (SG) smoothing filter function. This represents the total length of the SG smooth sliding window. This indicates the order of the polynomial used for SG smoothing. This indicates the velocity of the key point in the i-th frame. This indicates the velocity of the key point in the i-th frame. This indicates the preset SG smoothing time window length.

[0059] It should be noted that those skilled in the art can set the size of the preset SG smoothing time window according to actual needs, and this invention does not limit it.

[0060] In this embodiment of the invention, through two-stage robust preprocessing and smoothing derivative, keypoint jitter and outlier noise introduced by human pose estimation are effectively suppressed, providing a stable and reliable coordinate and velocity sequence for subsequent kinematic feature extraction, and significantly improving the signal-to-noise ratio and robustness of ground contact state discrimination.

[0061] Reference manual attached Figure 3 The diagram illustrates a physical quantity corresponding to four elements provided by an embodiment of the present invention.

[0062] Appendix Figure 3 The forward direction, ankle height off the ground, ankle velocity relative to the pelvis, and tibial angular velocity together constitute the core of the feature vector of this invention. It comprehensively and complementaryly describes the foot's motion state during the gait cycle from four dimensions: spatial position, vertical motion, horizontal motion, and rotational motion, providing robust and information-rich input for subsequent unsupervised probabilistic modeling.

[0063] S3: Based on smooth key point coordinates and velocity sequences, extract multiple kinematic features related to the two-foot ground contact state.

[0064] Among them, kinematic characteristics refer to physical quantities used to describe the motion state of an object, but do not involve the cause of the motion.

[0065] Optionally, kinematic characteristics include: ankle height off the ground, ankle velocity relative to the pelvis, and tibial angular velocity.

[0066] Among these, ankle height off the ground refers to the vertical distance of the ankle joint from the robustly estimated "ground reference height". Ankle velocity relative to the pelvis refers to the speed of movement of the ankle joint relative to the center point of the pelvis. Tibial angular velocity refers to the angular velocity of the lower leg (approximately represented by the knee-ankle line) rotating about its proximal joint (knee).

[0067] In one possible implementation, S3 specifically includes sub-steps S301 to S305: S301: Calculate the pelvic center point based on smoothed key point coordinates.

[0068] The pelvic center point refers to the midpoint of the line connecting the key points of the left and right hip joints.

[0069] Specifically, the method for calculating the center point of the pelvis is as follows: .

[0070] in, This represents the x-coordinate of the center point of the pelvis in the i-th frame image. This represents the x-coordinate of the left hip keypoint in the i-th frame. This represents the x-coordinate of the right hip keypoint in the i-th frame. This represents the y-coordinate of the center point of the pelvis in the i-th frame of the image. This represents the y-coordinate of the left hip keypoint in the i-th frame. This represents the y-coordinate of the right hip keypoint in the i-th frame.

[0071] S302: Based on the center point of the pelvis, calculate the ankle-relative pelvic velocity of the left and right ankle joints relative to the center of the pelvis.

[0072] Specifically, the ankle-to-pelvic velocity is calculated as follows: .

[0073] .

[0074] in, This represents the relative velocity of the left ankle in the x-direction relative to the center of the pelvis at frame i. This represents the absolute velocity of the left ankle keypoint in the x-direction at frame i. This represents the absolute velocity of the pelvic center point in the x-direction at frame i. This represents the relative velocity of the left ankle in the y-direction relative to the center of the pelvis at frame i. This represents the absolute velocity of the left ankle keypoint in the y-direction at frame i. This represents the absolute velocity of the pelvic center point in the y-direction at frame i. This represents the relative velocity of the right ankle in the x-direction relative to the center of the pelvis at frame i. This represents the absolute velocity of the right ankle keypoint in the x-direction at frame i. This represents the relative velocity of the right ankle in the y-direction relative to the center of the pelvis at frame i. This represents the absolute velocity of the right ankle keypoint in the y-direction at frame i.

[0075] S303: Calculate the ankle height above the ground for the left and right ankle joints based on smoothed key point coordinates.

[0076] Specifically, ankle clearance: Since the y-axis is positive downwards, the ground corresponds to a larger y-value. The tail quantiles of the y-sequences for both the left and right ankles are estimated, and the median of the tail quantiles is taken as the "ground reference height." .

[0077] .

[0078] in, This indicates the number of tail digits in the left ankle y-coordinate sequence. The function that calculates quantiles. This represents the y-coordinate of the left ankle keypoint in the i-th frame. This indicates the ground reference height that is robustly estimated for the left ankle trajectory. This indicates the number of tail digits in the right ankle y-coordinate sequence. This represents the y-coordinate of the left ankle keypoint in the i-th frame. This represents the ground reference height that is robustly estimated for the right ankle trajectory.

[0079] Among them, the tail percentile refers to the value that is located at a higher percentage position after a set of data is sorted from smallest to largest.

[0080] The tail median refers to the median of the y-coordinates calculated again from the filtered "tail" data.

[0081] Among them, the ground reference height refers to the y-coordinate value representing the "ground" in the video, which is estimated by the robust statistical method mentioned above.

[0082] And calculate the height above the ground: .

[0083] in, The preferred value is 0.90, which is robust to occasional downward jitter (sudden increase in y).

[0084] in, This represents the height of the left ankle off the ground in frame i. This represents the height of the right ankle off the ground in frame i.

[0085] S304: Calculate the tibial direction angles on the left and right sides based on the smoothed keypoint coordinates.

[0086] S305: Perform phase expansion and differentiation on the left and right tibial direction angles to obtain the left and right tibial angular velocities.

[0087] Optionally, the tibial angular velocity, specifically using the knee-ankle line as the approximate direction of the tibia, is as follows: .

[0088] .

[0089] right The angular velocity is obtained by performing phase unwrap and then smoothing the derivative. .

[0090] in, This represents the tibial orientation angle of the left leg in frame i. This represents the tibial direction angle of the right leg in frame i. This represents the y-coordinate of the key point on the left ankle. This represents the y-coordinate of a key point on the right ankle. This represents the y-coordinate of the key point on the left knee. This represents the y-coordinate of a key point on the right knee. The x-coordinate of the key point on the left ankle. This represents the x-coordinate of a key point on the right ankle. This represents the x-coordinate of a key point on the left knee. This represents the x-coordinate of a key point on the right knee. Indicates the angular velocity of the left tibia. This indicates the angular velocity of the right tibia.

[0091] In this embodiment of the invention, by comprehensively extracting multi-dimensional kinematic features such as ankle lift-off height, ankle relative pelvic velocity, and tibial angular velocity, more comprehensive and complementary feature evidence is provided for ground contact state recognition. This overcomes the shortcomings of single features being susceptible to noise or perspective, and significantly enhances the accuracy and robustness of state discrimination.

[0092] S4: Based on multiple kinematic features, the estimated direction of human movement is determined through a perspective-adaptive mechanism, and the foot velocity is projected onto the direction of human movement to obtain perspective-adaptive velocity features.

[0093] Among them, the viewpoint adaptive mechanism refers to an algorithm strategy that can automatically adapt to changes in the camera's shooting angle.

[0094] Among them, the estimation of the direction of progress refers to the estimation result of the direction of progress.

[0095] Velocity projection refers to the operation of mapping a vector to a scalar component in another specific direction.

[0096] It should be noted that existing technologies typically extract velocity features and set thresholds based on a fixed direction (such as the horizontal axis) in the image coordinate system. This can lead to feature distortion and recognition failure when the camera is not parallel to the walking direction. This invention robustly estimates the human body's forward direction from swing-phase motion data and projects the ankle joint's relative velocity onto this direction, achieving viewpoint normalization of motion features. This improvement enables the method to adapt to different shooting angles, fundamentally solving the problem of drastic performance degradation of traditional methods under oblique perspectives and significantly improving cross-scene applicability.

[0097] Specifically, in oblique perspectives or when the camera is not aligned with the treadmill / direction of travel, the primary forward and backward movements of the feet may not correspond to the image x-axis. This invention optionally estimates the direction of human movement and projects the relative velocity onto that direction.

[0098] In one possible implementation, S4 specifically includes sub-steps S401 to S407: S401: Select the time frames where the ankle height above the ground is greater than the left threshold and the right threshold, respectively, as the high-heeled frames during the swing period.

[0099] It should be noted that those skilled in the art can set the size of the left and right thresholds according to actual needs, and this invention does not limit this.

[0100] In this context, a time frame refers to a single image frame in a video, which is the smallest unit of sequence data and is identified by the frame index i.

[0101] Among them, the swing phase high-lift frames refer to the set of frames in which the foot is clearly lifted off the ground and is in the swing phase, based on the ankle lift height.

[0102] Specifically, the method for selecting samples during the swing period (high-off-ground frames) is as follows: .

[0103] in, This represents the set of frame indices i for which the left leg is determined to be in the swing phase (high off the ground). This represents the height of the left ankle off the ground in frame i. The function that calculates quantiles. This represents the set of frame indices i for which the right leg is determined to be in the swing phase (high off the ground). This represents the height of the right ankle off the ground in the i-th frame.

[0104] S402: Construct a relative velocity sample set based on the ankle-to-pelvis velocity corresponding to the high-ground frame during the swing phase.

[0105] The relative velocity sample set refers to the set of left and right ankle relative pelvic velocity vectors corresponding to all swing phase high-elevation frames.

[0106] Specifically, a relative velocity sample set is constructed: .

[0107] S403: By centering the median and pruning the MAD scale, a robust preprocessing of the relative velocity sample set is performed to obtain a standardized sample set.

[0108] Median centering refers to a data preprocessing step that subtracts the median of the dataset from each sample in the dataset across all dimensions, thereby making the data distribution centered around the median point.

[0109] MAD scaling refers to a robust outlier handling step that sets the data range for each dimension based on the median absolute deviation (MAD) and clips values ​​that exceed the range to the boundary values.

[0110] S404: Perform principal component analysis on the standardized sample set to obtain the direction of the first principal component, and use the direction of the first principal component as the estimate of the direction of human movement.

[0111] Principal component analysis is a classic statistical method for dimensionality reduction and feature extraction.

[0112] Here, the first principal component direction refers to the feature vector corresponding to the first principal component obtained by PCA.

[0113] S405: Calculate the explanatory variance ratio of the human body's forward direction estimate.

[0114] The explained variance ratio refers to the proportion of the variance of the first principal component to the total variance of all principal components.

[0115] Specifically, robust PCA seeks the main direction: First, median centering is applied and outliers are clipped according to the MAD scale. Then, SVD / PCA is used to obtain the first principal component direction d (unit vector), and forced... To ensure sign stability. The explained variance ratio is defined as: .

[0116] in, This indicates the proportion of variance explained by the first principal component. This represents the standard deviation of the first principal component in a PCA analysis. This represents the standard deviation of the second principal component in the PCA analysis.

[0117] S406: Determine if the explained variance ratio is greater than the reliability threshold. If yes, use the estimated human forward direction. Otherwise, use the default horizontal direction.

[0118] It should be noted that those skilled in the art can set the reliability threshold according to actual needs, and this invention does not limit it.

[0119] The default horizontal direction refers to the direction the system reverts to when the estimated direction of human movement is deemed unreliable.

[0120] S407: Project the relative velocity of the left and right ankles to the pelvis onto the selected direction to obtain the velocity characteristics.

[0121] Specifically, when (Preferred value 0.55) is considered a reliable estimate of the human body's forward direction. Velocity projection: If the human body's forward direction is reliable, then the relative velocity along the forward direction is defined as follows: .

[0122] .

[0123] Otherwise, rollback will be used. As a characteristic of horizontal relative velocity.

[0124] in, This represents the projection of the relative velocity of the left ankle onto the estimated forward direction (the direction of human movement) at frame i. This represents the projection of the relative velocity of the right ankle onto the estimated forward direction (the direction of human movement) at frame i. This represents the component of the unit vector d along the x-axis of the image coordinate system. This represents the component of the unit vector d along the y-axis of the image coordinate system. This represents a general representation, referring to the feature used when the estimation of the human body's forward direction is unreliable, that is, directly using the original relative velocity in the x-direction.

[0125] In this embodiment of the invention, by adaptively estimating the direction of human movement from the data and projecting the velocity, the motion characteristics are transformed from an image coordinate system susceptible to viewpoint distortion to a direction of movement with clear physical meaning. This solves the core problem of the failure of the fixed threshold method under oblique viewpoints and significantly improves the robustness and generalization ability of the method to different shooting angles.

[0126] S5: Based on multiple kinematic features and perspective-adaptive velocity features, an unsupervised two-component Gaussian mixture model is used to model the state of both feet touching the ground and calculate the posterior probability that both feet are in the support phase.

[0127] Among them, the unsupervised two-component Gaussian mixture model refers to a probabilistic model that learns the intrinsic structure of data from the data itself without requiring manual labeling.

[0128] In the bipedal support phase gait cycle, this is the stage where both feet are in contact with the ground simultaneously.

[0129] In Bayesian statistics, posterior probability refers to the conditional probability of an event (such as "the foot is in the support phase") occurring after observing the current data (feature vector).

[0130] It should be noted that existing methods often rely on manually setting fixed thresholds or require large amounts of labeled data to train classifiers, making it difficult to adapt to individual gait differences, speed variations, and different wearing conditions. This invention, targeting a single video sequence, utilizes an unsupervised Gaussian mixture model to automatically learn the probability distributions of the support and swing phases in the feature space, without any prior labeling or threshold tuning. This improvement achieves fully data-driven adaptive modeling, avoiding overfitting and cross-domain generalization problems, and enhancing the method's practicality and robustness in real-world, variable environments.

[0131] Specifically, for a single test sequence, this invention constructs a feature vector for each foot and uses an unsupervised two-component GMM to learn the probability distribution boundary of the support phase / swing phase, in order to avoid the need for fixed thresholds and cross-domain supervised training.

[0132] In one possible implementation, S5 specifically includes sub-steps S501 to S505: S501: Construct the feature vectors of the left foot and the right foot.

[0133] The left foot feature vector is a multidimensional feature representation constructed from the left foot to describe the foot's motion state in the current frame. The right foot feature vector is a multidimensional feature representation constructed from the right foot to describe the foot's motion state in the current frame.

[0134] Specifically, the feature vector is: .

[0135] in, This represents the feature vector of the left foot in the i-th frame. This represents the horizontal velocity feature of the left foot used for modeling in frame i. for or .

[0136] S502: Robustly standardize each dimension of the left and right foot feature vectors to obtain standardized left and right foot feature vectors.

[0137] Specifically, robust standardization: calculates the median for each feature dimension d. and The scale and standardized features are obtained: .

[0138] And cut to To suppress extreme values.

[0139] in, Represents the robust scaling estimate of the d-th dimension feature. This represents the absolute deviation of the median of the d-th eigenvalues ​​across all samples. To represent a very small positive number, This represents the standardized value of the d-th dimension feature in the i-th frame. This represents the d-th dimension of the original feature value in the i-th frame. This represents the median of the d-th eigenvalues ​​of all samples. Let represent the robust scaling estimate of the d-th dimension feature.

[0140] S503: The standardized left-foot eigenvector and the standardized right-foot eigenvector are modeled using an unsupervised two-component Gaussian mixture model, and the model parameters are estimated iteratively using the expectation-maximization algorithm.

[0141] Among them, the expectation-maximization algorithm refers to an iterative optimization algorithm used to estimate the parameters of a model containing latent variables.

[0142] Here, model parameters refer to the parameters to be learned in the Gaussian mixture model.

[0143] Specifically, a two-component diagonal Gaussian mixture: The parameters are (in (This is a diagonal matrix). EM iteration is used: .

[0144] .

[0145] .

[0146] in, Let represent the posterior responsibility of the i-th sample belonging to the k-th component. This represents the mixing weight of the k-th Gaussian component. The probability density function represents the multivariate Gaussian distribution. Let represent the mean vector of the k-th Gaussian component. Let the covariance matrix of the k-th Gaussian component be denoted as . This represents the mixing weight of the l-th Gaussian component. Let represent the mean vector of the l-th Gaussian component. Let the covariance matrix of the l-th Gaussian component be denoted as . This represents the number of "soft" samples belonging to the k-th component in the current sequence. This indicates the total number of samples (total frames) in the current sequence. This represents a small positive regularization term.

[0147] S504: Based on the model parameters and the characteristics of the support phase, select the left foot support phase component and the right foot support phase component.

[0148] Among them, the characteristics of the support phase refer to the statistical regularities exhibited by the support phase in the characteristic space. The support phase typically corresponds to: a small height above the ground, a small vertical velocity, a small horizontal velocity, and a small angular velocity.

[0149] Among them, the left foot support phase component refers to the component that corresponds to the "support phase" automatically identified from the components of the left foot GMM through an algorithm.

[0150] Among them, the right foot support phase component refers to the component that corresponds to the "support phase" automatically identified from the components of the right foot GMM through an algorithm.

[0151] S505: Based on the left foot support phase component and the right foot support phase component, calculate the posterior probability that the left foot is in the support phase and the posterior probability that the right foot is in the support phase.

[0152] Specifically, the selection of the support phase component and its posterior probability: In z-space, the support phase typically has a small height above the ground and a small relative velocity / angular velocity. This is to automatically determine the "support component". Left foot .

[0153] .

[0154] .

[0155] When there are insufficient available samples or features are unavailable, p can degenerate to 0.5 (uncertain). The same applies to the right foot.

[0156] in, This represents the support phase score of the k-th component. This indicates the weight of the ground clearance in the scoring function. This represents the mean of the feature dimension corresponding to the ground altitude in the mean vector of the k-th component. This indicates the weight of vertical velocity in the scoring function. This represents the mean of the vertical velocity absolute value feature dimension in the mean vector of the k-th component. This indicates the weight of the horizontal velocity feature in the scoring function. This represents the mean of the horizontal velocity feature dimension in the mean vector of the k-th component. This indicates the weight of angular velocity in the scoring function. This represents the mean of the feature dimension corresponding to the absolute value of the angular velocity in the mean vector of the k-th component. Indicates the index of the component selected as the supporting phase. This indicates that the scoring function The component index k that yields the minimum value This represents the posterior probability of the left foot's support phase in frame i. This indicates that the i-th sample belongs to the i-th sample. Posterior responsibility of each component.

[0157] In this embodiment of the invention, by performing unsupervised probabilistic modeling on a single sequence segment, the characteristic distributions of the support phase and the oscillation phase are automatically learned and separated, fundamentally eliminating the dependence on fixed threshold parameter tuning or a large amount of labeled data, realizing adaptation to individual differences and velocity changes, and significantly improving the generalization and practicality of the method.

[0158] S6: Based on the posterior probability, the two-foot-touching state is jointly temporally decoded using a four-state coupled hidden Markov model and the Viterbi decoding algorithm to obtain a temporally consistent two-foot-touching state sequence.

[0159] The four-state coupled Hidden Markov Model (HMM) is a statistical model used for state modeling and inference of time series. The "HMM" assumes that the system state (here, the two-footed-to-the-ground state) is an unobservable "hidden state," but can be indirectly inferred from observations (here, posterior probabilities). "Four states" refers to the value space of the hidden states. "Coupled" emphasizes that the model models the left and right foot states as a whole, rather than treating them as two separate HMMs.

[0160] The Viterbi decoding algorithm refers to a dynamic programming algorithm used in Hidden Markov Models (HMMs). Given an observation sequence (posterior probabilities of all frames) and model parameters (transition probabilities, emission probabilities), it finds the hidden state sequence most likely to have generated that observation sequence, i.e., the maximum a posteriori (MAP) state sequence.

[0161] It should be noted that existing technologies often process the left and right foot states independently or only use temporal smoothing post-processing, ignoring the physical coupling relationship between the two (such as the alternation of single and double support), which can easily lead to state inconsistencies or temporal jitter. This invention jointly models the two foot ground-touching states as a four-state coupled Hidden Markov Model, embeds gait periodic constraints through the state transition matrix, and uses Viterbi decoding to obtain the globally optimal state sequence. This improvement ensures that the output sequence is not only temporally smooth but also strictly conforms to the biomechanical laws of gait. Furthermore, minimum duration frame filtering further suppresses noise, comprehensively improving the physical rationality and temporal consistency of the recognition results.

[0162] Reference manual attached Figure 2 The diagram illustrates a four-state coupled HMM state diagram and a schematic diagram of the main transition relationships provided by an embodiment of the present invention.

[0163] The constructed four-state coupled hidden Markov model (HMM) defines four combinations of the two-foot-on-the-ground state: 00 (both feet off the ground), 10 (left foot on the ground), 01 (right foot on the ground), and 11 (both feet on the ground). Its state transition diagram encodes the basic physical laws of gait—single support and double support must alternate, and there is usually no direct jump from two feet off the ground to two feet on the ground.

[0164] Specifically, there is coupling between the left and right foot contact states (single support, double support, flight phase), therefore a four-state coupled HMM pair is constructed. Joint modeling: .

[0165] In one possible implementation, S6 specifically includes sub-steps S601 to S605: S601: Construct the observation probability matrix of a four-state coupled hidden Markov model based on posterior probability.

[0166] In an Hidden Markov Model (HMM), the observation probability matrix is ​​a matrix that describes the probability that each hidden state "produces" various observations.

[0167] Specifically, the observation / emission probability: using Construct the emission probability for each state : .

[0168] .

[0169] .

[0170] .

[0171] in, This represents the firing probability when the left foot is on the ground and the right foot is off the ground. This represents the posterior probability of the right foot's support phase in frame i. This represents the firing probability when the right foot is on the ground and the left foot is off the ground. This represents the emission probability when both feet are on the ground. This represents the emission probability when both feet are off the ground. Represents an exponential function. This represents the penalty coefficient for the state where both feet are off the ground.

[0172] in To suppress the state where both feet leave the ground simultaneously (which is rarely seen in walking scenarios), it is preferable to perform calculations in the logarithmic domain to improve numerical stability.

[0173] S602: Set the state transition probability matrix of the four-state coupled hidden Markov model.

[0174] The state transition probability matrix is ​​a matrix that describes the probability of transitions between hidden states over time.

[0175] Specifically, the transition probability: Let the self-maintaining probability be... (0.80-0.9999, preferably 0.985). Based on this, the main transitions are restricted to patterns that conform to gait patterns (e.g., single support → double support → opposite side single support), and unreasonable transitions are assigned an extremely low probability.

[0176] S603: Based on the observation probability matrix and the state transition probability matrix, the four-state coupled hidden Markov model is decoded using the Viterbi decoding algorithm to obtain the maximum a posteriori state sequence.

[0177] Among them, the maximum a posteriori state sequence refers to the sequence with the highest posterior probability among all possible state sequences found by the Viterbi decoding algorithm.

[0178] S604: Map the maximum a posteriori state sequence to the initial left foot touching the ground state sequence and the initial right foot touching the ground state sequence.

[0179] The initial left foot-touching state sequence refers to the preliminary binary sequence obtained when the maximum a posteriori state sequence is mapped back to the independent representation of the left foot.

[0180] The initial right foot ground-touching state sequence refers to the preliminary binary sequence obtained when the maximum a posteriori state sequence is mapped back to the independent representation of the right foot.

[0181] Specifically, Viterbi decoding involves finding the maximum a posteriori state sequence using Viterbi dynamic programming. And obtain the binary touchdown sequence: .

[0182] in, This represents the binary ground-touching state sequence of the left foot after decoding. Indicates an indicator function, This indicates the decoding status of the i-th frame. This represents the binary ground-touching state sequence of the right foot after decoding.

[0183] S605: Perform minimum duration frame filtering on the initial left foot touching state sequence and the initial right foot touching state sequence to obtain a left foot touching state sequence and a right foot touching state sequence with consistent timing.

[0184] Among them, minimum sustained frame number filtering refers to a simple post-processing noise reduction method. It scans the binary state sequence, finds all segments that are consecutively 1 (grounding), and if the length (number of frames) of a segment is less than the minimum sustained frame number threshold, it is considered to be a "pseudo-grounding" caused by keypoint jitter, and all segments of that segment are set to 0.

[0185] Specifically, minimum sustained frame count decluttering: for Perform minimum connected segment length filtering separately; if the ground contact segment length is less than... Then set it to 0 to further eliminate the transient "false ground touch" caused by keypoint noise. This represents the minimum number of consecutive frames threshold.

[0186] It should be noted that those skilled in the art can set the minimum continuous frame rate threshold according to actual needs, and this invention does not limit it.

[0187] In this embodiment of the invention, by using a four-state coupled Hidden Markov Model and Viterbi decoding, the single-frame probability estimation is integrated with temporal constraints and gait physics, and transient noise is filtered out. Finally, a physically reasonable, temporally smooth and robust bipedal ground-touching state sequence is output, which significantly improves the reliability and practicality of the recognition results.

[0188] S7: Based on the bipedal ground contact state sequence, the bipedal ground contact state is identified by extracting ground contact events and ground lift events.

[0189] The ground contact event refers to the instant during the gait cycle when the foot first makes contact with the ground, usually called heel strike.

[0190] The ground-lift event refers to the moment when the foot leaves the ground during the gait cycle, usually called toe-off.

[0191] In one possible implementation, S7 specifically includes sub-steps S701 to S703: S701: Calculate the frame difference between the two-foot-touching state sequence.

[0192] Among them, frame difference refers to a simple and effective method for detecting abrupt changes in a sequence. In this invention, first-order discrete differences are calculated for the binary grounding state sequences of the left and right feet respectively.

[0193] S702: Based on the difference between the previous and next frames, extract the ground contact event and the ground lift event of both feet respectively.

[0194] Specifically, the ground contact sequence for each foot Define the difference: .

[0195] when The event is determined to be a ground contact event (HS). The event is determined to be an off-ground event (TO), and the corresponding frame number (frame_idx) and timestamp are output. .

[0196] in, This represents the difference between the preceding and following frames for the ground-touching state of the i-th frame. This represents the binary ground-touching state of a foot (left or right) in the i-th frame. This represents the binary grounding state of the same foot in frame i-1.

[0197] S703: The identified ground contact and ground lift events, along with the corresponding frame index and timestamp, are integrated into a gait event list output to complete the identification of bipedal ground contact status.

[0198] The gait event list refers to a structured output data table that summarizes all identified event information.

[0199] Specifically, the output should include at least: the posterior probabilities of left and right foot support phases. Binary ground contact sequences of left and right feet A list of ground contact / lift-off events. And intermediate quantities used for debugging and quality control (ground clearance, relative velocity, angular velocity, direction of human movement and its explained variance ratio, etc.).

[0200] In this embodiment of the invention, a smooth ground-touch state sequence is accurately converted into discrete gait event points (ground touch and ground lift) through differential detection, and a complete output containing multi-granularity information (probability, state sequence, event list) is provided, directly meeting the terminal requirements for gait analysis and parameter calculation, and completing the fully automated recognition process from video to structured gait information.

[0201] In practical applications, key point sequences of human posture are extracted from monocular videos and subjected to robust preprocessing and smoothing differentiation to suppress noise and obtain stable motion trajectories. Multi-dimensional kinematic features such as ankle lift-off height, ankle-to-pelvis velocity, and tibial angular velocity are extracted, and a perspective-adaptive mechanism is introduced to estimate the human forward direction, projecting the features onto physically meaningful directions. An unsupervised Gaussian mixture model is used to adaptively model the single sequence to obtain the posterior probability of the support phase. Furthermore, a four-state Hidden Markov Model coupling the left and right foot states is constructed, combined with Viterbi decoding and temporal filtering to obtain a smooth bi-foot ground contact sequence that conforms to gait patterns. Finally, ground contact and lift-off events are extracted, and complete recognition results are output. This invention does not rely on wearable sensors, force platforms, or manually labeled data, achieving low-cost, non-contact foot-ground contact analysis. Through perspective adaptation and intra-sequence unsupervised learning, the generalization ability to different shooting angles, individual differences, and movement speeds is significantly improved. The use of multi-feature fusion and coupled temporal modeling enhances the robustness, physical plausibility, and temporal consistency of the recognition results. The final output contains multi-level information including probability estimates, state sequences, and event lists, which can directly support gait parameter calculations and biomechanical analysis, and has high practical value.

[0202] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention effectively solves the recognition failure problem in existing technologies caused by fixed thresholds failing to adapt to different shooting angles, individual differences, and walking speeds, through a perspective adaptation mechanism and single-sequence unsupervised probabilistic modeling. It can automatically correct motion features under complex shooting conditions such as oblique perspectives and adaptively learn state boundaries for each input sequence, without requiring manual parameter tuning or reliance on labeled data. This significantly improves the generalization ability, robustness, and practicality of monocular video foot contact state recognition methods in different real-world scenarios.

[0203] Reference manual attached Figure 4 The diagram shows a schematic of the structure of a bipedal foot contact state recognition system based on monocular video provided by the present invention.

[0204] The present invention also provides a bipedal foot-ground contact state recognition system 20 based on monocular video, applied to the above-mentioned bipedal foot-ground contact state recognition method based on monocular video, comprising: Processor 201.

[0205] The memory 202 stores computer-readable instructions that, when executed by the processor 201, implement the bipedal foot contact state recognition method based on monocular video as described in the method embodiment.

[0206] The bipedal foot-to-ground contact state recognition system 20 based on monocular video provided by the present invention can perform the above-described bipedal foot-to-ground contact state recognition method based on monocular video and achieve the same or similar technical effects. To avoid repetition, the present invention will not elaborate further.

[0207] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0208] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0209] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. A computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. Semiconductor media can be solid-state drives.

[0210] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0211] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0212] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0213] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0214] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0215] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0217] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0218] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0219] This invention provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the bipedal foot contact state recognition method based on monocular video as described in the method embodiment.

[0220] The present invention provides a computer-readable storage medium that can implement the steps and effects of the bipedal foot contact state recognition method based on monocular video in the above-described method embodiments. To avoid repetition, the present invention will not repeat the details.

[0221] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0222] The following points need to be explained: (1) The accompanying drawings of the embodiments of the present invention only involve the structures involved in the embodiments of the present invention. Other structures can refer to the general design.

[0223] (2) For clarity, the thickness of layers or regions is enlarged or reduced in the drawings used to describe embodiments of the invention, i.e., these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being “above” or “below” another element, the element may be “directly” located “above” or “below” the other element or there may be intermediate elements.

[0224] (3) Where there is no conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other to obtain new embodiments.

[0225] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A monocular video-based bipedal foot-ground contact state recognition method, characterized in that, include: S1: Obtain the 2D keypoint sequence of human pose in monocular video; S2: Perform robust outlier suppression and smoothing differentiation on the two-dimensional keypoint sequence to obtain a smoothed keypoint coordinate and velocity sequence; S3: Based on the coordinates of the smoothed key points and the velocity sequence, extract multiple kinematic features related to the bipedal ground contact state; S4: Based on the multiple kinematic features, the estimated direction of human movement is determined through a perspective adaptation mechanism, and the foot velocity is projected based on the direction of human movement to obtain perspective-adaptive velocity features. S5: Based on the multiple kinematic features and the viewpoint adaptive velocity features, the state of the two feet touching the ground is modeled using an unsupervised two-component Gaussian mixture model, and the posterior probability of the two feet being in the support phase is calculated. S6: Based on the posterior probability, the two-footed ground-touching state is jointly temporally decoded using a four-state coupled hidden Markov model and the Viterbi decoding algorithm to obtain a temporally consistent two-footed ground-touching state sequence. S7: Based on the bipedal ground contact state sequence, the bipedal ground contact state is identified by extracting the ground contact event and the ground lift event.

2. The monocular video-based bipedal foot contact state recognition method according to claim 1, characterized in that, The following is included after S1 and before S2: S1A: Obtain the time information corresponding to the two-dimensional keypoint sequence; wherein, the time information includes frame index, timestamp, and scale factor; S1B: Based on the time information, convert the pixel coordinates of the two-dimensional keypoint sequence into metric coordinates.

3. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, S2 specifically includes: S201: Calculate the median and MAD of the two-dimensional keypoint sequence within the sliding window using the Hampel robust filtering algorithm; S202: Replace the outliers in the two-dimensional keypoint sequence according to the median and the MAD to obtain a robust coordinate sequence; S203: The robust coordinate sequence is smoothed using the SG smoothing algorithm to obtain the smoothed key point coordinates; S204: Take the first derivative of the coordinates of the smoothed key points to obtain the velocity sequence.

4. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, The kinematic characteristics include: ankle height off the ground, ankle velocity relative to the pelvis, and tibial angular velocity.

5. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, S3 specifically includes: S301: Calculate the pelvic center point based on the coordinates of the smoothing key points; S302: Based on the center point of the pelvis, calculate the ankle-relative pelvic velocity of the left and right ankle joints relative to the center of the pelvis; S303: Calculate the ankle height off the ground of the left and right ankle joints based on the coordinates of the smooth key points; S304: Calculate the tibial direction angles on the left and right sides based on the coordinates of the smoothing key points; S305: Perform phase expansion and differentiation on the left and right tibial direction angles to obtain the tibial angular velocities on the left and right sides.

6. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, S4 specifically includes: S401: Select the time frames on the left and right sides where the ankle height above the ground is greater than the left threshold and the right threshold, respectively, as high-heel-off frames during the swing period; S402: Construct a relative velocity sample set based on the ankle-to-pelvis velocity corresponding to the high-ground frame during the swing phase; S403: Robust preprocessing of the relative velocity sample set is performed by median centering and MAD scale pruning to obtain a standardized sample set; S404: Perform principal component analysis on the standardized sample set to obtain the direction of the first principal component, and use the direction of the first principal component as the estimated direction of the human body's forward movement; S405: Calculate the explained variance ratio of the estimated human body forward direction; S406: Determine whether the explained variance ratio is greater than the reliability threshold; if so, use the estimated human forward direction; otherwise, use the default horizontal direction. S407: Project the relative velocity of the ankles on the left and right sides to the selected direction to obtain the velocity characteristics.

7. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, S5 specifically includes: S501: Construct the feature vectors of the left foot and the right foot; S502: Perform robust standardization on each dimension of the left foot feature vector and the right foot feature vector to obtain the standardized left foot feature vector and the standardized right foot feature vector. S503: The standardized left-foot feature vector and the standardized right-foot feature vector are modeled using the unsupervised two-component Gaussian mixture model, and the model parameters are estimated iteratively using the expectation-maximization algorithm. S504: Based on the model parameters, select the left foot support phase component and the right foot support phase component through the support phase characteristics; S505: Based on the left foot support phase component and the right foot support phase component, calculate the posterior probability of the left foot being in the support phase and the posterior probability of the right foot being in the support phase.

8. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, S6 specifically includes: S601: Based on the posterior probability, construct the observation probability matrix of the four-state coupled hidden Markov model; S602: Set the state transition probability matrix of the four-state coupled hidden Markov model; S603: Based on the observation probability matrix and the state transition probability matrix, the four-state coupled hidden Markov model is decoded using the Viterbi decoding algorithm to obtain the maximum a posteriori state sequence; S604: Map the maximum a posteriori state sequence to the initial left foot touching the ground state sequence and the initial right foot touching the ground state sequence; S605: Perform minimum duration frame filtering on the initial left foot touching state sequence and the initial right foot touching state sequence to obtain a left foot touching state sequence and a right foot touching state sequence with consistent timing.

9. The method for identifying bipedal foot-to-ground contact state based on monocular video according to claim 1, characterized in that, Specifically, S7 includes: S701: Calculate the frame difference between the preceding and following frames of the bipedal ground-touching state sequence; S702: Based on the difference between the preceding and following frames, extract the ground contact event and the ground lift event of both feet respectively; S703: The identified ground contact event and ground lift event, along with the corresponding frame index and timestamp, are integrated into a gait event list for output, thereby completing the identification of the bipedal ground contact state.

10. A bipedal foot contact state recognition system based on monocular video, characterized in that, include: processor; A memory storing computer-readable instructions, which, when executed by the processor, implement the bipedal foot contact state recognition method based on monocular video as described in any one of claims 1 to 9.

Images