A Segmented Evaluation Method Based on Markov Models
By establishing a unified time reference to align human posture and ball trajectory sequences, and combining switched Kalman filtering and hierarchical hidden Markov models, the unstable segmentation problem of volleyball hitting actions was solved, resulting in more accurate calculation of specific indicators and interpretable scoring results.
Patent Information
- Application Number
- CN202610805227.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-05
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2046-06-05
AI Technical Summary
Existing technologies exhibit instability in locating key events and segmenting stages in volleyball hitting motions. Alignment between human posture sequences and ball trajectory sequences is difficult, and there is a lack of sequence and duration constraints, resulting in inconsistent segmentation results and difficulty in outputting interpretable specific indicators and scores.
A unified time reference is established to align the human posture sequence with the ball trajectory sequence. The linear dynamic system and the hierarchical hidden Markov model are switched by contact probability modulation. Iterative coupling inference is performed by combining switched Kalman filtering. The output includes the ball hitting action type, contact time, release time and stage segmentation results. Specific indicators are calculated to generate scores and error correction prompts.
This improved the stability of critical event detection and the consistency of segmented results, enhanced the interpretability and reproducibility of the evaluation, and ensured the accuracy and consistency of the calculation of specific indicators.
Smart Images

Figure CN122598264B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of sports motion assessment, and more particularly to a method for evaluating ball strike segments based on Markov models. Background Technology
[0002] In the teaching and training of volleyball hitting actions such as serving, passing, setting, and spiking, it is often necessary to analyze the action process in segments and extract specific technical indicators to achieve quantitative evaluation and error correction guidance. With the development of computer vision and machine learning technologies, existing technologies typically use monocular or multi-view video to acquire motion process data, extract human posture sequences through human keypoint detection models, and obtain the ball trajectory sequence using target detection and tracking methods. Based on this, some solutions further employ threshold-based rule-based methods, classifier-based event detection methods, or state estimation methods such as Kalman filtering to smooth and predict the ball trajectory; other solutions use temporal models such as Hidden Markov Models to identify action categories or stages, thereby achieving a certain degree of automated evaluation.
[0003] However, the aforementioned existing technologies still have shortcomings in locating key events and segmenting stages of volleyball hitting motions, mainly in the following aspects:
[0004] 1. Critical event moments (such as the moment of ball contact and the moment of ball release) often rely on a single clue (distance threshold, speed change, or manual annotation). In cases of occlusion, detection jitter, frame rate changes, or missing ball trajectory, they are prone to false detection or moment drift, making it difficult to reproduce stably.
[0005] 2. Human posture sequences and ball trajectory sequences are often obtained from different models or sampling mechanisms, resulting in problems such as inconsistent timestamps, missing values, and noise accumulation. This makes it difficult to align the two in time, which in turn affects the accuracy of contact judgment, stage boundary positioning, and subsequent index calculation.
[0006] 3. The action segmentation often adopts independent classification or segmentation strategies, lacking unified modeling of prior constraints such as the sequence and duration of the stages, and also lacking the coupling utilization between changes in the ball's motion state and the stage state. This easily leads to discontinuous segmentation, stage jumps, or difficulty in outputting interpretable specific indicators and scoring results corresponding to the technical actions.
[0007] Therefore, a method for evaluating shot segments is needed to address the shortcomings of existing technologies. Summary of the Invention
[0008] One objective of this invention is to propose a segmented evaluation method for volleyball hitting based on a Markov model. Addressing the problems in existing technologies such as the difficulty in stably locating key moments in volleyball hitting, the inaccurate alignment of human posture sequences and ball trajectory sequences, and the lack of sequential and duration constraints in automatic segmentation of action phases, making it difficult to output interpretable specific indicators and scores, this invention proposes a method that establishes a unified time reference and aligns human posture sequences and ball trajectory sequences; calculates contact candidate features to generate contact probability sequences; utilizes contact probability to modulate the mode state transition of a switching linear dynamic system and performs switched Kalman filtering to obtain ball state estimates and posterior probabilities of mode states; further, it constructs a hierarchical Hidden Markov model containing contact anchor state and release anchor state and applies stage transition constraints and duration constraints; and iteratively couples the switching linear dynamic system and the hierarchical Hidden Markov model to infer and output the action type, contact moment, release moment, and segmentation results. Based on this, it calculates specific indicators such as ball throw offset, hitting point height, forearm platform angle, and peak arm swing acceleration to generate scores and error correction prompts. This invention offers the technical advantages of more stable key event detection, stronger consistency in segmentation results, and interpretable and reproducible evaluation indicators.
[0009] This invention provides a method for evaluating shot segments based on Markov models, including:
[0010] S1. Acquire video data of the same volleyball hitting action and establish a unified time reference; S2. Extract human posture sequence and ball trajectory sequence under the unified time reference and align them in time; S3. Calculate contact candidate features based on the time-aligned human posture sequence and ball trajectory sequence, and generate a contact probability sequence from the contact probability model; S4. Construct a switching linear dynamic system containing mode states based on the ball trajectory sequence, determine the transition probability of the mode state based on the contact probability sequence, and perform switching Kalman filtering to obtain the ball state estimation sequence and the posterior probability sequence of the mode state; S5. Based on the human posture sequence and ball state estimation... A hierarchical hidden Markov model is constructed using sequence, contact probability sequence, and posterior probability sequence of mode state. The lower-level stage states include the contact anchor state and the release anchor state. Stage state transition constraints and duration constraints are set. Through iterative inference, the lower-level stage states impose constraints on the mode state transition of the switching linear dynamic system. The posterior probability sequence of mode state is used as the observation evidence of the hierarchical hidden Markov model to update the stage states. The output includes the hitting action type, contact time, release time, and the stage segmentation results of the hitting action. S6. Based on the stage segmentation results of the hitting action, the preset special indicators are calculated, and the scoring results and error correction prompts are output.
[0011] Optionally, S1 includes:
[0012] When acquiring the video data, the acquisition timestamps corresponding to each frame of the video data are recorded, and the frames are sorted according to the acquisition timestamps to generate the image sequence.
[0013] A unified time base is established based on the collection timestamp, and the collection timestamp is mapped to a time index under the unified time base to generate the time index sequence;
[0014] When acquiring sensor data while acquiring video data, the sampling timestamp of the sensor data is recorded, and the sampling timestamp is mapped to the unified time base, so that the time index sequence corresponds to the sensor data under the unified time base.
[0015] Optionally, S2 includes:
[0016] The image sequence is processed frame by frame according to the time index sequence. The human key point detection model is used to output the two-dimensional or three-dimensional coordinates of the human key points at each time index. The two-dimensional or three-dimensional coordinates of the human key points at each time index are arranged according to a unified time reference to generate the human pose sequence.
[0017] The image sequence is processed frame by frame according to the time index sequence. The sphere target detection model is used to output the two-dimensional or three-dimensional coordinates of the sphere at each time index. The two-dimensional or three-dimensional coordinates of the sphere at each time index are arranged according to a unified time reference to generate the sphere trajectory sequence.
[0018] When the human posture sequence or the ball trajectory sequence has missing values at some time indices, interpolation is performed to complete the missing values.
[0019] When the sampling intervals of the human posture sequence and the ball trajectory sequence are inconsistent, the human posture sequence and the ball trajectory sequence are resampled or interpolated based on the time index sequence, so that the human posture sequence and the ball trajectory sequence correspond one-to-one at each time index, thereby obtaining the time-aligned human posture sequence and the time-aligned ball trajectory sequence.
[0020] Optionally, S3 includes:
[0021] At each time index, based on the human body key points in the time-aligned human posture sequence and the two-dimensional or three-dimensional coordinates of the ball in the time-aligned ball trajectory sequence, the relative distance between the two-dimensional or three-dimensional coordinates of the ball and the two-dimensional or three-dimensional coordinates of the human body key points corresponding to the hand or forearm is calculated.
[0022] Based on the two-dimensional or three-dimensional coordinates of the ball at adjacent time indices and the two-dimensional or three-dimensional coordinates of the key points of the human body corresponding to the hand or forearm, calculate the relative velocity between the ball and the hand or forearm.
[0023] Calculate the velocity or acceleration of the hand or forearm based on the two-dimensional or three-dimensional coordinates of the key points of the human body corresponding to the hand or forearm at adjacent time indices;
[0024] Calculate the change in velocity or acceleration of the ball's trajectory based on the two-dimensional or three-dimensional coordinates of the ball at adjacent time indices;
[0025] The relative distance, relative velocity, velocity or acceleration of the hand or forearm, and the change in velocity or acceleration of the ball trajectory are used to form contact candidate features. Before inputting the contact candidate features into the contact probability model, normalization or standardization processing is performed on the contact candidate features. The contact probability model is a probability mapping model based on training parameters or a preset threshold. The contact probability model includes one of the following: logistic regression model, Naive Bayes model, support vector machine probabilistic model, neural network classification model, and threshold-based segmented mapping model. The contact candidate features are mapped to contact probabilities through the contact probability model. The contact probability is a probability value between 0 and 1, and the contact probabilities at each time index are arranged according to a unified time base to generate the contact probability sequence.
[0026] Optionally, S4 includes:
[0027] Based on the time-aligned ball trajectory sequence, a switching linear dynamic system is constructed to describe the ball's motion. The switching linear dynamic system consists of ball state, mode state, linear state transition model, and observation model. The ball state includes the ball's position and velocity, and the observation model is used to map the ball state to observations corresponding to the ball trajectory sequence.
[0028] Set corresponding linear state transition model parameters and process noise parameters for different mode states;
[0029] The conditional transition probability of the mode state is determined based on the contact probability sequence, such that the transition probability from the mode state corresponding to non-contact to the mode state corresponding to contact increases as the contact probability increases, and the transition probability from the mode state corresponding to contact to the mode state corresponding to non-contact increases as the contact probability decreases.
[0030] The conditional transition probability is determined by the contact probability at the time index through a monotonic function, which includes a linear truncation function, a sigmoid function, a softmax function, or a piecewise threshold function. The time-varying mode state transition matrix is constructed from the conditional transition probability.
[0031] Under the conditional transition probability constraint, a switched Kalman filter is performed on the ball trajectory sequence to generate the ball state estimation sequence and the mode state posterior probability sequence.
[0032] Based on the mode state posterior probability sequence, determine the time index at which the dominant mode changes at adjacent time indices, or determine the time index at which the contact corresponding mode state posterior probability exceeds a preset threshold, and arrange the determined time indices according to a unified time base to generate the mode switching candidate time sequence.
[0033] Furthermore, when determining the conditional transition probability of the mode state based on the contact probability sequence, a first threshold for entering the mode state corresponding to contact and a second threshold for exiting the mode state corresponding to contact are set, and the first threshold is greater than the second threshold. In addition, a minimum dwell time constraint is set for the mode state corresponding to contact, so that when the duration of the mode state corresponding to contact is less than the minimum dwell time, it is not allowed to transition to the mode state corresponding to non-contact.
[0034] Furthermore, the mode states include free flight mode state, hand contact mode state, and forearm contact mode state, and different linear state transition model parameters and process noise parameters are set for the hand contact mode state and the forearm contact mode state, respectively.
[0035] Optionally, S5 includes:
[0036] A hierarchical hidden Markov model is established based on the time-aligned human posture sequence, the ball state estimation sequence, the contact probability sequence, and the mode switching candidate time sequence. In the hierarchical hidden Markov model, an upper-level state set is set to represent the type of hitting action and a lower-level state set is set to represent the stage of the hitting action.
[0037] In the lower-level state set, a ball-touching anchor state and a release anchor state are set, and the contact probability sequence and the mode switching candidate time sequence are used to construct observational evidence of the ball-touching anchor state and the release anchor state.
[0038] The observations of the hierarchical hidden Markov model include human posture features, ball state features, contact probability, and posterior probability of the pattern state.
[0039] The stage states in the lower-level state set are limited to transition in a preset order by a preset stage state transition constraint, and the duration of each stage state in the lower-level state set is limited under a unified time reference by a preset duration constraint; the correspondence between the stage states in the lower-level state set and the mode states of the switching linear dynamics system is established in advance to determine the set of mode states allowed for the mode state transition of the switching linear dynamics system.
[0040] The correspondence includes one of the following: a preset mapping table, a mapping model learned based on training data, or a rule mapping based on a contact probability threshold;
[0041] In the non-contact corresponding phase state, restrict the mode state transition from entering the contact corresponding mode state, or limit the transition probability to enter the contact corresponding mode state to below a preset upper limit;
[0042] In the contact anchor state and the release anchor state, the mode state is allowed to transition between the contact-corresponding mode state and the non-contact-corresponding mode state; when inferring the hierarchical hidden Markov model, the lower-level state is used to limit the set of mode states allowed for the mode state transition of the switched linear dynamics system, and the posterior probability sequence of the mode state is used as observation evidence for the hierarchical hidden Markov model to participate in the state posterior probability update; the iterative update of the switched linear dynamics system and the hierarchical hidden Markov model includes alternately performing the following steps: (a) under the condition of fixing the lower-level state posterior probability or the maximum a posteriori path of the hierarchical hidden Markov model, performing switched Kalman filtering or switched Kalman filtering on the switched linear dynamics system. (a) Smoothing process to update the ball state estimation sequence and the pattern state posterior probability sequence; (b) Under the condition of fixing the pattern state posterior probability sequence, perform forward-backward inference processing or maximum a posteriori path decoding processing on the hierarchical hidden Markov model to update the state posterior probability of the lower-level state, and update the state posterior probabilities of the ball-touching anchor state and the ball-leaving anchor state; (c) Use the update results of steps (a) and (b) as the observation evidence or constraint input for the next iteration; Under the condition of satisfying the stage state transition constraint and the duration constraint, iteratively update the switching linear dynamic system and the hierarchical hidden Markov model until a preset convergence condition is met, the preset convergence condition including one or a combination of the following:
[0043] The number of iterations has reached the preset limit;
[0044] The increment of the log-likelihood value between two consecutive iterations is less than a preset threshold;
[0045] The time index corresponding to the maximum a posteriori of the ball-touching anchor state or the ball-leaving anchor state remains unchanged during a preset number of iterations.
[0046] The maximum a posteriori path decoding outputs the ball-hitting action type identification result, the ball-hitting time, the ball-leaving time, and the ball-hitting action stage segmentation result arranged according to a unified time reference.
[0047] Furthermore, the observational evidence for the ball-touching anchor state and the release-from-hand anchor state is obtained by fusing multiple pieces of evidence, including: local maxima of the contact probability sequence, local maxima of the rate of change of ball velocity in the ball state estimation sequence, and local maxima of hand or forearm acceleration in the human posture sequence. The weighted sum or product of these multiple pieces of evidence is used as the observational likelihood of the anchor state to update the posterior probability of the anchor state.
[0048] Optionally, S6 includes:
[0049] Based on the segmentation results of the ball-hitting action phase, the time interval corresponding to each ball-hitting action phase is determined, and the anchor point time point corresponding to the ball-hitting anchor point state and the ball-leaving anchor point state is determined at the time index where the ball-hitting moment and the ball-leaving moment are located.
[0050] The horizontal position, vertical position, and horizontal reference direction are determined based on a preset coordinate system, which includes one of the following: an image coordinate system, wherein the first axis is the horizontal axis and the second axis is the vertical axis;
[0051] The site coordinate system established by camera calibration has the vertical axis aligned with the direction of gravity.
[0052] A coordinate system established based on the direction of gravity determined by inertial sensors;
[0053] Within the time interval of the ball-throwing action phase, the ball-throwing offset is calculated by the offset of the ball's horizontal position in the ball state estimation sequence relative to the horizontal position of the key points of the human body corresponding to the human torso.
[0054] At the anchor point time point corresponding to the ball contact anchor point state, the height of the hitting point is calculated based on the vertical position of the ball in the ball state estimation sequence;
[0055] During the time interval of the hitting action phase corresponding to the ball setting or passing, the forearm platform angle is calculated by the angle between the line connecting the key points of the human body corresponding to the forearm and the horizontal reference direction.
[0056] Within the time interval of the ball-striking action phase corresponding to the arm-swing acceleration, the peak arm-swing acceleration is calculated based on the maximum value of the acceleration or angular velocity of the key points of the human body corresponding to the hand; the ball-throwing deviation, the ball-striking point height, the forearm platform angle, and the peak arm-swing acceleration are compared with the preset reference range to obtain the deviation of each specific indicator;
[0057] The scoring result is calculated based on the deviation using a scoring mapping function. The scoring result is a score value or a score level. The scoring mapping function includes one of the following: a linear mapping function, a piecewise mapping function, a sigmoid function, or a weighted summation function. Error correction suggestions are generated based on the specific indicators whose deviation exceeds a preset threshold and their deviation direction. The error correction suggestions are then associated with the ball contact time, the ball release time, or the segmented results of the ball striking action phase and output.
[0058] The beneficial effects of this invention are:
[0059] 1. By establishing a unified time reference and aligning human posture sequences with ball trajectory sequences, and combining missing value interpolation and resampling processing, the impact of time misalignment and data gaps caused by different sampling mechanisms and detection jitter is reduced, thereby improving the accuracy and consistency of subsequent contact judgment, event localization, and index calculation.
[0060] 2. By generating contact probability sequences through contact candidate features, and modulating the mode state transition probability of the switching linear dynamic system with contact probability, combined with switching Kalman filtering to obtain ball state estimation and mode posterior probability, key events such as ball contact and release can still be stably detected under noise, occlusion or trajectory discontinuity, reducing false detection and time drift.
[0061] 3. By using the posterior probability of the ball's motion pattern as observational evidence for the hierarchical Hidden Markov Model, and introducing the contact anchor state and release anchor state, stage sequence constraints, and duration constraints in the lower stage, a dual-model iterative coupling inference is adopted to obtain coherent stage segmentation results that conform to the movement law. This allows for the reliable calculation of specific indicators such as ball throw deviation, hitting point height, forearm platform angle, and peak arm swing acceleration, and the output of scores and error correction prompts, thereby improving the interpretability and reproducibility of the evaluation. Attached Figure Description
[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0063] Figure 1 The flowchart shows the shot segmentation evaluation method based on the Markov model.
[0064] Figure 2 This is a flowchart of step S5 of the present invention. Detailed Implementation
[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0066] refer to Figures 1-2 The shot segmentation evaluation method based on Markov models includes:
[0067] S1. Acquire video data of the same volleyball hitting action and establish a unified time reference; S2. Extract human posture sequence and ball trajectory sequence under the unified time reference and align them in time; S3. Calculate contact candidate features based on the time-aligned human posture sequence and ball trajectory sequence, and generate a contact probability sequence from the contact probability model; S4. Construct a switching linear dynamic system containing mode states based on the ball trajectory sequence, determine the transition probability of the mode state based on the contact probability sequence, and perform switching Kalman filtering to obtain the ball state estimation sequence and the posterior probability sequence of the mode state; S5. Based on the human posture sequence and ball state estimation... A hierarchical hidden Markov model is constructed using sequence, contact probability sequence, and posterior probability sequence of mode state. The lower-level stage states include the contact anchor state and the release anchor state. Stage state transition constraints and duration constraints are set. Through iterative inference, the lower-level stage states impose constraints on the mode state transition of the switching linear dynamic system. The posterior probability sequence of mode state is used as the observation evidence of the hierarchical hidden Markov model to update the stage states. The output includes the hitting action type, contact time, release time, and the stage segmentation results of the hitting action. S6. Based on the stage segmentation results of the hitting action, the preset special indicators are calculated, and the scoring results and error correction prompts are output.
[0068] In this specific embodiment, S1 includes:
[0069] When acquiring video data of the same volleyball hitting action, the camera acquisition interface simultaneously calls the same system monotonic clock to obtain the acquisition timestamp corresponding to each frame of the image, and records the timestamp of the frame as the [number of frames]. Frame image is Its collection timestamp is ,in The frame number is a non-negative integer. For the first Image data of the frame, To and A one-to-one timestamp, with the unit being seconds;
[0070] All frames are sorted in ascending order of timestamp to generate an image sequence. And simultaneously rearrange the corresponding timestamp sequence When identical timestamps exist, the original collection order is maintained to ensure stable sorting.
[0071] Using the timestamp of the first frame after sorting as the reference time for a unified time base, denoted as... ,in The zero point of a unified time base and satisfying ;
[0072] The time step, denoted as the median difference between the timestamps of adjacent frames, is used as the unified time reference. ,in The time interval between adjacent time indices is based on a unified time base and is expressed in seconds, and is derived from a sequence. The median was determined to suppress the impact of acquisition jitter on the time step;
[0073] The acquisition timestamp of each frame is mapped to a time index under a unified time base to generate a time index sequence. The mapping relationship is expressed by the formula:
[0074] ;
[0075] in In order to be with the first The discrete-time index corresponding to the frame, and it is a non-negative integer. The rounding operator is used to round down, and 0.5 is used to perform the rounding function. For the first Frame capture timestamp For reference time, For time step;
[0076] Indexed by time The key is used to establish a mapping table from time index to image frame, so that image frame can be directly accessed by time index under a unified time base. At the same time, the key of the mapping table is arranged in ascending order to obtain the time index sequence for subsequent steps to process by index.
[0077] While acquiring video data and sensor data simultaneously, the same system monotonic clock is synchronously invoked for each sensor sampling record to obtain the sampling timestamp of that record, and this timestamp is recorded as follows: The sensor sample is Its sampling timestamp is ,in The sample number is a non-negative integer. This is the sensor's original observation vector, containing sampled values from each channel. To and A one-to-one timestamp, with the unit being seconds;
[0078] Use the same as video frames and Will Mapped to a unified time index corresponding to the sensor A mapping table from time index to sensor observation is established so that the time index sequence corresponds to the sensor data index by index under a unified time reference. When multiple sensor observations appear at the same time index, these observations are arithmetically averaged across channels to obtain the fused observation for that time index. When a certain time index lacks sensor observations, a missing marker is recorded at that time index position for subsequent interpolation or resampling processing.
[0079] In this specific embodiment, S2 includes:
[0080] Based on time index sequence For image sequences Frame-by-frame processing is performed, and a time-series container of uniform length is constructed using the time index as the primary key. A global time index set is defined as follows. ,in It is the set of all discrete-time indices under a unified time base. The total number of time indices and satisfying For each frame of image Write it to the time index For the corresponding slot, when multiple frame images correspond to the same time index, the acquisition timestamp is selected. With the center time of this time index The frame with the smallest absolute difference is used as the unique representative frame for that time index, where For frame acquisition timestamps, A reference time for unifying time standards, To standardize the time step for a unified time base;
[0081] Index at each time At this point, the representative frame image corresponding to the time index is input into the human keypoint detection model. Output the set of human body key point coordinates and its confidence set ,in This is a keypoint regression model based on deep neural networks. The network parameters are trained and fixed offline for the model. The number of key points on the human body and the value is [value]. For time index First Two-dimensional coordinates of key points These are the horizontal pixel coordinates in the image coordinate system, with the unit being pixels. These are the vertical pixel coordinates in the image coordinate system, with the unit being pixels. For the first The confidence level of each key point and the range of values is: When the confidence level of a certain key point The coordinates of this key point are then marked as missing values;
[0082] index the same time All key point coordinates are numbered according to key point number. The fixed sequence of concatenation forms the human pose vector. ,in For time index The human pose vector at the location is used to construct the human pose sequence. ;
[0083] Index at the same time At this point, the frame image will be input into the sphere target detection model. Output sphere bounding box and its confidence level ,in This is a target detection model based on deep neural networks. The network parameters are trained and fixed offline for the model. The coordinates of the top-left corner of the bounding box are in pixels. The width of the bounding box is in pixels. The bounding box height is in pixels. To detect confidence level and the value range is ,when The spherical coordinates at that time index are then marked as missing values.
[0084] Let the center of the bounding box be the two-dimensional coordinates of the sphere and denoted as:
[0085] ;
[0086] in The horizontal pixel coordinates of the sphere are in pixels. Let be the vertical pixel coordinates of the sphere, in pixels. The ball trajectory sequence is obtained by arranging the time indexes.
[0087] When a human pose sequence or ball trajectory sequence has missing values at some time indices, linear interpolation is used to complete each missing segment, and the consecutive missing values at the beginning and end of the segment are extended using the nearest valid value to ensure that each time index has a definite value. Linear interpolation applies to any coordinate component. Formula used:
[0088] ;
[0089] in Indicates time index The coordinate component to be completed and Taken from one of or Or taken from of or The nearest known time index to the left of the missing segment. The index of the nearest known time to the right of the missing segment. For time index The known coordinate components at that location, For time index The known coordinate components at the location;
[0090] Through the above Based on index-wise organization and interpolation completion, human pose sequences are made more accurate. With ball trajectory sequence At each time index, a time-aligned sequence of human poses and a time-aligned sequence of ball trajectories are obtained.
[0091] In this specific embodiment, S3 includes:
[0092] Time-aligned human pose sequences Time-aligned ball trajectory sequence Calculate contact candidate features and generate a contact probability sequence, where A set of time indices under a unified time base. The total number of time indices. The number of key points in the human body. For time index First Two-dimensional coordinates of key points For time index The two-dimensional coordinates of the sphere The unified time step determined for S1;
[0093] Index at each time The key point of the left elbow is selected from the key points of the human body. Key points of the right elbow Key points of the left wrist Key points of the right wrist Calculate the Euclidean distance from the ball to the left and right wrists, select the side with the smaller distance as the contact side for this time index, and record the wrist key point on that side as the hand key point. And record the key point of the elbow on that side as the elbow key point. ,in Two-dimensional coordinates of key hand points. Two-dimensional coordinates of key points at the elbow;
[0094] based on and The key points of constructing the forearm are ,in The coordinates of the midpoint of the forearm are two-dimensional, used to characterize the spatial position of the forearm in contact with the ball;
[0095] In time index Calculate the relative distance characteristics between the ball and the hand or forearm. ,in , For a two-dimensional Euclidean norm, Using spherical coordinates, The coordinates of key points on the hand Coordinates of key points on the forearm;
[0096] Calculate the ball velocity vector based on the position difference at adjacent time indices. With hand velocity vector The Euclidean norm of the difference vector between the two is used as the characteristic of relative velocity. ,in For the ball in time index The velocity vector at that point, For hands in time index The velocity vector at that point, The scalar value representing the relative velocity between the ball and the hand;
[0097] Calculate the hand acceleration amplitude characteristics based on the hand velocity difference at adjacent time indices. ,in This is a scalar value representing the amplitude of hand acceleration.
[0098] Calculate the ball acceleration vector based on the ball velocity difference at adjacent time indices. The magnitude of the change in ball acceleration is used as a characteristic of the ball's motion. ,in For the ball in time index The acceleration vector at that point, The magnitude of the change in the ball's acceleration is a scalar value.
[0099] when season ,when season To ensure that the feature values at all time indices are determined;
[0100] The above four scalar features are arranged in a fixed order to form a time index. Contact candidate feature vectors at the location ,in It is a four-dimensional column vector with fixed meaning and order of its components;
[0101] Standardization is performed on the contact candidate feature vectors to eliminate scale differences, using the mean vector obtained statistically from the labeled dataset during the offline training phase. with standard deviation vector right Perform a dimension-wise transformation to obtain the standardized eigenvectors ,in It is a four-dimensional mean column vector. It is a four-dimensional column vector of standard deviations, with the standard deviation of each dimension being greater than zero. It is a four-dimensional standardized column vector;
[0102] The contact probability model uses a logistic regression model and standardized feature vectors. Input / output time index Contact probability at point The mapping relationship is as follows:
[0103] ;
[0104] in The contact probability is given, and its value range is given. , It is an exponential function. It is a four-dimensional weight column vector. Transpose it. For bias scalars, It is a four-dimensional standardized feature column vector;
[0105] The parameters of the logistic regression model and During offline training, minimize the bandwidth. The binary cross-entropy loss of the regularization term is determined and stored in a fixed manner, where the regularization coefficient takes the value of . Furthermore, the parameters are not updated after training, allowing the online inference phase to obtain each time index with only one vector dot product and one exponentiation operation. ;
[0106] The contact probabilities at all time indices are arranged in order according to a unified time base to obtain the contact probability sequence. It also outputs the constraints for mode state transitions and the observational evidence for the ball-touching anchor state and the release anchor state for subsequent steps.
[0107] In this specific embodiment, S4 includes:
[0108] Based on time-aligned ball trajectory sequence With contact probability sequence Construct a switched linear dynamical system including mode states and perform a switched Kalman filter, where A set of time indices under a unified time base. The total number of time indices. (Time index) The two-dimensional pixel coordinates of the sphere, The coordinates are horizontal pixel coordinates, and the unit is pixels. These are vertical pixel coordinates, with the unit being pixels. For time step;
[0109] Define the ball state as And define the observation as ,in For time index The column vector of the ball's state. The speed is horizontal and the unit is pixels per second. The vertical velocity is expressed in pixels per second. For time index The observed column vector is given by Direct construction;
[0110] Define the mode state as And respectively represent the free flight mode, hand contact mode, and forearm contact mode, where For time index Discrete mode label;
[0111] The following state equations and observation equations are used to switch the linear dynamic system, and different state transition model parameters and process noise parameters are configured for different mode states:
[0112] ;
[0113] in For mode state The chosen state transition matrix, For the observation matrix, It is process noise and follows a zero-mean Gaussian distribution. For mode state The chosen process noise covariance matrix To observe noise and satisfy a zero-mean Gaussian distribution To observe the noise covariance matrix, It is a multivariate Gaussian distribution. It is a zero vector;
[0114] Fix the observation matrix as So that the observation corresponds only to the position component;
[0115] The state transition matrix corresponding to the free flight mode state is fixed as follows: And fix its process noise covariance as ,in This represents a diagonal matrix construction operator where the first two terms on the diagonal are in units of pixel squares and the last two terms are in units of pixels per second squares.
[0116] The state transition matrix corresponding to the hand contact mode state is fixed as follows: And fix its process noise covariance as ,in It is the velocity decay coefficient in the hand contact mode and is used to describe the dynamic characteristics of the ball's rapid velocity change under the action of the hand;
[0117] The state transition matrix corresponding to the forearm contact mode state is fixed as follows: And fix its process noise covariance as ,in It is the velocity attenuation coefficient in the forearm contact mode and is used to describe the dynamic characteristics of the ball under the action of the forearm platform, where the velocity change is relatively smooth but still significantly deviates from free flight.
[0118] The observation noise covariance is fixed as and ,in Let $\begin{p}$ be the standard deviation of the pixel positioning error for ball detection, expressed in pixels. It is a second-order identity matrix;
[0119] At the initialization time, the mean of the initial state is fixed as The initial state covariance is fixed at 1. ,in This is the initial estimate of the ball's state at time index 0. The initial uncertainty of the ball's state at time index 0;
[0120] Constructing a time-varying mode state transition matrix based on the contact probability sequence To implement hysteresis constraints with different entry and exit thresholds, and to superimpose a minimum dwell time constraint, where From the pattern Transfer to mode The conditional transition probability, with the entry threshold fixed at... The exit threshold is fixed at And satisfy The minimum stay time is fixed as A time index;
[0121] To determine the specific mode state corresponding to entering contact, key hand points will be used to construct the contact side. Key points of the forearm Introducing pattern allocation discrimination, if The corresponding contact mode will be assigned to the hand contact mode state; otherwise, it will be assigned to the forearm contact mode state. For a two-dimensional Euclidean norm, For time index Coordinates of key points on the hand. For time index Coordinates of the midpoint of the forearm;
[0122] The total transition probability of entering contact is The time is fixed at 0.90, in The value is fixed at 0.05. The time is fixed at 0.50, and the transition probability from the free flight mode to the corresponding contact mode is set accordingly. The remaining probability;
[0123] The total transition probability of exiting contact is only allowed to take effect after the dwell count meets the minimum dwell time constraint. Specifically, the dwell count is defined. For time index The number of time indices that have been continuously in the corresponding contact mode state up to date, of which It is a non-negative integer and in Increasing over time, Reset to zero when The total transition probability of exiting contact is fixed at 0 to prevent the transition from the contact-corresponding mode state to the free flight mode state. and The total transition probability of exiting contact is fixed at 0.90. and The total transition probability of exiting contact is fixed at 0.05, and settings are based on this. and and the corresponding self-maintaining probability and Meanwhile, the direct transition probability between different contact modes is fixed to 0 to suppress frequent changes in contact type caused by detection noise;
[0124] In the above Under constraints, a three-mode switching Kalman filter is executed. The filtering process employs an interactive multi-model hybrid prediction and update mechanism, and indexes at each time step. The output shows the filtered state estimates and covariance of each mode. Simultaneously, based on observational innovation and innovation covariance, the Gaussian observation likelihood of each mode is calculated and combined. The posterior probability of the mode state is obtained by recursively normalizing the prior of the mode:
[0125] ;
[0126] in For time index Pre-condition mode The posterior probability, For time index 0 to The observed sequence is used to finally obtain the sphere state estimate by weighting and fusing the filtered state estimates of each mode with the posterior probability of the mode. The ball state estimation sequence is obtained by arranging the sequences according to their time indices. With the posterior probability sequence of the mode state ;
[0127] A sequence of candidate time points for mode switching is generated based on the posterior probability sequence of mode states. Specifically, the dominant mode label is calculated. And in satisfying Time Index Record the candidate switching time and calculate the total posterior probability of contact. And set a threshold ,when From less than Become greater than or equal to Record the contact entry candidate time, when From greater than or equal to Become less than Record the contact exit candidate time, arrange the time index of the above records in order according to a unified time base to generate a sequence of mode switching candidate times and output them.
[0128] In this specific embodiment, S5 includes:
[0129] Based on time-aligned human pose sequence and sphere state estimation sequence Contact probability sequence , mode state posterior probability sequence And constructing a hierarchical hidden Markov model from the candidate time sequences of mode switching and performing iterative inference, where A set of time indices under a unified time base. The total number of time indices. For time index The column vector for estimating the state of the ball is given, and its position and velocity components represent the two-dimensional position and two-dimensional velocity of the ball, respectively. For time index The probability of contact at a given location and the range of values are: For time index In mode state The posterior probability, The mode state defined for S4 and Indicates free flight mode status, Indicates the hand contact mode status, Indicates the forearm contact mode status. For time index 0 to Observation sequence of the Earth;
[0130] In a hierarchical Hidden Markov Model, the upper-level states represent the type of hitting action and are fixed as a single action type that remains unchanged throughout the entire sequence. The action type label is defined as follows. These represent serve, pass, dig, and spike, respectively. The lower-level states represent the hitting action phase and are defined as follows: ,in For time index The stage status labels are as follows: Stage 1 represents the preparation stage, Stage 2 represents the guidance stage, Stage 3 represents the arm swing acceleration stage, Stage 4 represents the ball contact anchor point state, Stage 5 represents the release anchor point state, and Stage 6 represents the follow-up stage.
[0131] To ensure that the segmented results conform to the action rules, the state transition constraints of the lower-level stages are fixed to only allow actions according to the rules. Sequential transitions are allowed, and each stage is self-sustaining; cross-stage transitions are not allowed, and reverse transitions are not allowed.
[0132] To implement duration constraints, the lower-level stages are modeled as Hidden Markov Processes with explicit durations, and a duration range under a unified time base is given for each stage. The duration of stage 1 is 5 to 60 time indices, the duration of stage 2 is 3 to 40 time indices, the duration of stage 3 is 2 to 30 time indices, the duration of stage 4 is 1 to 3 time indices, the duration of stage 5 is 1 to 3 time indices, and the duration of stage 6 is 5 to 60 time indices. Duration constraints are implemented by enumerating only the candidate durations that fall within the corresponding range during dynamic programming decoding.
[0133] To construct observational evidence, indexing is performed at each time step. Generate observation vector at location And as an observation for the lower stage, in which , For the velocity component in the ball state estimation The calculated ball velocity amplitude, The first difference of the ball velocity amplitude divided by the time step The amplitude of the rate of change of ball velocity and The unified time step determined for S1, The acceleration amplitude of the hand or forearm is obtained by second-order difference of the coordinates of key points of the human body under a unified time reference, and the larger of the two values is taken to characterize the sudden change of the limb at the moment of contact.
[0134] For non-anchor stage The observation probabilities are calculated using a diagonal Gaussian distribution associated with the action type and stored in a parameter table obtained from offline training. Specifically, for each action type... With each non-anchor stage Store the mean vector With the diagonal covariance vector ,in and The sample mean and variance are calculated on corresponding subsets of training sequences with stage and action type annotations, and a lower bound is applied to the variance of each dimension. To avoid numerical degradation;
[0135] The anchor point observation likelihoods for the contact anchor point state and the release anchor point state are obtained through multi-evidence fusion and used as the observation probabilities for the anchor point stage. This is first derived from the contact probability sequence. Extract local maxima and require that the local maxima satisfy the following conditions: Secondly, from the sequence of the amplitude of the ball velocity change rate Extract local maxima and require that the local maxima satisfy the following conditions: Again, from the acceleration amplitude sequence of the hand or forearm Extract local maxima and require that the local maxima satisfy the following conditions: The determination of local maxima adopts the following method: The neighborhood of the two time indices to the left and right of the center is compared, and the corresponding evidence is set to zero when the local maximum or threshold conditions are not met.
[0136] The three types of evidence were normalized to... The anchor point observation likelihood is then obtained by weighted summation. The calculation formula is as follows:
[0137] ;
[0138] in For time index The likelihood observed at the anchor point and the range of values is The contact probability evidence weight is set to a value of 1. The evidence weight for the rate of change of ball velocity is given, and its value is [value missing]. The evidence weight for hand or forearm acceleration is set to a value of 1. To be The numerical value directly used as evidence of normalized contact probability To be Divide by a constant And cut off to The obtained numerical value of the normalized rate of change of ball velocity evidence, To be Divide by a constant And cut off to The obtained normalized acceleration evidence values;
[0139] The candidate time sequence for mode switching is used to limit the set of candidate time indices for anchor points. Specifically, it is calculated and retained only within a window of five time indices to the left and right of any candidate time. As valid observational evidence of the contact anchor point state and the release anchor point state, the area outside the window... Set to zero to suppress false triggering during non-contact periods;
[0140] To implement the constraints of the lower-level stage on the mode transition of the switching linear dynamical system, a correspondence between the stage state and the mode state is pre-established and used as a hard constraint in the iterative inference, where when Only mode state is allowed at this time Appearance and entry prohibited ,when Allowed mode state and The transitions between these modes cover the moment of contact with the ball and the moment of release.
[0141] Iterative inference employs alternating updates and a fixed maximum number of iterations. In the 0th iteration, we directly use and Construct observation vector And in each action type The posterior probabilities of the stage states are obtained by performing forward and backward inference with explicit duration constraints. Then, the stage sequence is obtained by decoding the maximum a posteriori path of dynamic programming that satisfies both the stage sequence constraints and the duration constraints. The action type with the highest log-likelihood over the entire segment is used as the action type recognition result. ;
[0142] No. The step (a) of the round iteration is to sequence the stages. The process involves converting the mode state transition matrix in S4 into a reachability mask that varies with time, setting disallowed mode transition probabilities to zero, and then normalizing the result. This allows for the re-execution of the switched Kalman filter under stage constraints to update the process. and ;
[0143] No. Step (b) of the round iteration uses the updated and Reconstruct and anchor point observation likelihood sequence Then, forward-backward inference and maximum a posteriori path decoding are performed again to update the stage sequence. ;
[0144] When the maximum a posteriori time index of the ball-touching anchor state and the maximum a posteriori time index of the ball-leaving anchor state remain unchanged for two consecutive rounds of iteration, convergence is determined and the iteration is stopped early.
[0145] During the output phase, the condition is satisfied. The time index with the highest posterior probability in the set of time indices is used as the time of ball contact. Take the satisfied The time index with the highest posterior probability in the set of time indices is taken as the time of departure. ,by The stage number change position at adjacent time indices is used as the stage boundary to output the stage segmentation result of the hitting action, and the action type recognition result is also output simultaneously. Provided for S6 to perform specific indicator calculations and score generation.
[0146] In this specific embodiment, S6 includes:
[0147] Based on action type recognition results Time index corresponding to the moment of ball contact Time index corresponding to the moment of release and the results of the segmentation of the hitting action phase. Calculate specific indicators and output scoring results and error correction prompts, among which A set of time indices under a unified time base. The total number of time indices. For time index The stage status label at the point of contact, and the time point of the ball contact anchor are defined as time indexes. Corresponding unified time base time The time point at which the hand anchor is removed is defined as the time index. Corresponding unified time base time ,in To unify the reference time base, To standardize the time step;
[0148] The coordinate system adopted is the image coordinate system and is consistent with S2 and S4, with the horizontal axis being... The axis is positive to the right, and the vertical axis is... The axis is positive downwards. To ensure that the vertical position increases with "higher", the vertical position of the sphere is defined using... It means that, among them State estimation for S4 ball The vertical position component is in pixels;
[0149] Extract the time interval for each stage based on the segmentation results, and satisfy the following: All time indices are merged into stages based on continuity. The set of closed intervals and selecting those containing That segment is taken as the valid phase interval for this shot, thus obtaining the preparatory phase interval. Guiding phase interval Arm swing acceleration phase interval With follow-up stage interval ,in All are consecutive integer intervals with time indices;
[0150] To ensure comparability of metrics across different heights and image scales, shoulder width was used for normalization. The calibrated shoulder width value is defined as follows: And take the interval Inside The median, of which This is the shoulder width calibration value in pixels. For time index The horizontal pixel coordinates of the key point on the left shoulder For time index The horizontal pixel coordinates of the key point on the right shoulder are shown, and the key point number is the same as S2.
[0151] Ball throw offset indicator only In the case of calculation and corresponding serving action, the horizontal position of the torso is defined as And take the average of the horizontal coordinates of the key points on the left and right hips, that is ,in The horizontal pixel coordinates of the key point on the left hip are: The horizontal pixel coordinates of the key point on the right hip;
[0152] During the guidance phase The internal calculation of the sphere's horizontal offset relative to the torso for each time index The maximum absolute value is taken as the original amount of ball displacement. ,in The horizontal position component of the S4 sphere state estimate is given in pixels. This represents the original offset of the ball, expressed in pixels.
[0153] Normalizing the ball throw offset yields ,in It is a dimensionless quantity and represents the ball's trajectory deviation.
[0154] The impact point height index is calculated for all action types, in the time index of the ball contact anchor point. Take the ball in the vertical position Vertical position of the torso The difference is used as the original value of the height of the hitting point relative to the torso. ,in The vertical pixel coordinates of the torso at the moment of contact with the ball anchor point. The vertical pixel coordinates of the key point on the left hip. The vertical pixel coordinates of the right hip key point. The original height of the hitting point is expressed in pixels.
[0155] Normalizing the height of the hitting point yields ,in It is a dimensionless quantity representing the height of the hitting point.
[0156] Forearm platform angle index only Calculate the time index of the ball contact point, corresponding to the passing and volleying actions. The contact side is determined based on the distance from the ball to the left and right wrists, and the key point of the wrist on that side is determined accordingly. And the key point of the elbow is ;
[0157] In the interval The forearm direction vector is calculated for each time index. The platform angle is obtained by taking the arithmetic mean of the angles relative to the horizontal reference direction, taking the absolute values of the angles, and then calculating the arithmetic mean. ,in The unit is degrees and the horizontal reference direction is defined as the image coordinate system. Positive direction of the axis;
[0158] Peak arm swing acceleration is only available in Calculate the corresponding serve and spike actions under the given conditions, during the arm swing acceleration phase. Key points of contact with the inner side of the hand Two-dimensional position sequences according to a uniform time step The second-order difference is used to obtain the hand acceleration vector at each time index, and its Euclidean norm is calculated as the hand acceleration amplitude. ,in The unit is pixels per second squared;
[0159] Take interval Inside The maximum value is used as the raw peak value of the arm swing acceleration. and normalize it to obtain ,in This represents the original peak value of the arm swing acceleration. It is a dimensionless index of the peak arm swing acceleration.
[0160] The deviations of each specific indicator are compared with the preset reference range, and the reference range is then fixed. , When an indicator falls within the corresponding reference range, the deviation of that indicator is recorded as 0. When the indicator is less than the lower limit of the reference range, the deviation is recorded as the difference between the lower limit and the indicator. When the indicator is greater than the upper limit of the reference range, the deviation is recorded as the difference between the indicator and the upper limit.
[0161] Map the deviation to a rating value and output the rating level, denoted as . And the range of values is The rating levels are divided into thresholds. For A, For B, For C, For D, the score is calculated using a weighted deduction scoring mapping function as follows:
[0162] ;
[0163] in Number the indicator and Corresponding ball throw offset index Corresponding hitting point height index, Corresponding forearm platform angle index Corresponding to the peak arm swing acceleration index, For applicability indicators and when this indicator needs to be calculated for this type of action. otherwise The indicator weight is fixed at 1. For the first The deviation of each indicator is consistent with the above-mentioned deviation definition. For the first The deduction tolerance for each indicator is fixed at [value]. Operators that take smaller values;
[0164] Based on the indicators whose deviation exceeds the deduction tolerance and their deviation direction, error correction suggestions are generated and output in conjunction with anchor points or stage intervals. Corresponding deviation exceeds Output "Ball throw horizontal offset too large" and associate it with the interval ,when Corresponding deviation exceeds Furthermore, when the index is below the lower limit of the reference range, the output "hitting point is too low" is displayed and associated with the contact anchor point time. ,when Corresponding deviation exceeds Furthermore, when the indicator exceeds the upper limit of the reference range, the output "hitting point is too high" is displayed and associated with the contact anchor point time. ,when Corresponding deviation exceeds Furthermore, when the indicator exceeds the upper limit of the reference range, the output "Forearm platform angle is too large" is displayed and associated with the interval. ,when Corresponding deviation exceeds Furthermore, when the indicator is less than the lower bound of the reference range, the output "insufficient arm swing acceleration" is displayed and associated with the interval. ,when Corresponding deviation exceeds Furthermore, when the indicator exceeds the upper limit of the reference range, the output message "Excessive arm swing acceleration leads to decreased stability" is displayed and associated with the interval. and the rating value Rating levels, error correction suggestions, and ball contact anchor point timing. Time point of departure from anchor point The results of each stage and segment are output together to form the final evaluation result.
[0165] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
[0166] This invention addresses the technical problems in volleyball hitting instruction assessment, such as the difficulty in stably locating the contact and release points, the inability to automatically segment movement phases, and the lack of consistency in output indicators. It aligns the human posture sequence and the ball trajectory sequence under a unified time reference, and constructs contact candidate features based on their relative distance, relative velocity, hand or forearm movement changes, and ball motion changes, forming a contact probability sequence that changes over time. This contact probability sequence is further used to modulate the mode state transition of a switching linear dynamic system, enabling stable estimation of the dynamic differences between free flight and contact action using a switched Kalman filter, thus outputting a reliable ball state estimate and posterior probability of the mode state. Based on this, a hierarchical hidden Markov model uses the posterior probability of the ball motion mode and human posture features as observational evidence, segmenting the action according to phase sequence and duration constraints, and providing contact and release anchor points. This achieves coordinated stability between key event detection and action segmentation, and provides consistent time anchoring for the calculation of specific indicators such as ball throw deviation, contact point height, forearm platform angle, and peak arm swing acceleration, thereby achieving reproducible scoring and error correction prompts.
[0167] In terms of algorithm structure, this invention addresses the aforementioned technical problems by improving contact events: First, it introduces a contact-driven mode switching prior, making the mode state transition probability monotonically change with the contact probability, and allows setting entry and exit thresholds and minimum dwell time constraints to suppress frequent false switching caused by detection noise from the source; Second, it establishes a bidirectional coupled iterative inference mechanism between the switching linear dynamic system and the hierarchical hidden Markov model, using the stage state to inversely constrain the reachable set of ball motion modes, while using the mode posterior to inversely enhance the stage state update, ensuring that key events such as ball contact and release remain stable under conditions of occlusion, missed detection, and trajectory discontinuity; Third, it sets ball contact anchor state and release anchor state in the hierarchical hidden Markov model and applies duration constraints, fusing multi-source evidence such as contact probability peak, ball velocity change rate peak, and hand or forearm acceleration peak when necessary, improving the robustness of anchor point positioning and the coherence of segmentation results, thereby better achieving the technical effect of action segmentation evaluation.
Claims
1. A method for evaluating shot segments based on Markov models, characterized in that, include: S1. Acquire video data of the same volleyball hitting action and establish a unified time reference; S2. Extract the human posture sequence and ball trajectory sequence under a unified time reference, and align the two in time. S3. Calculate contact candidate features based on time-aligned human posture sequences and ball trajectory sequences, and generate contact probability sequences from the contact probability model; S4. Construct a switching linear dynamic system containing mode states based on the ball trajectory sequence, determine the transition probability of the mode states based on the contact probability sequence, and perform a switching Kalman filter to obtain the ball state estimation sequence and the posterior probability sequence of the mode states. S5. Construct a hierarchical hidden Markov model based on human posture sequence, ball state estimation sequence, contact probability sequence and pattern state posterior probability sequence, and set an upper-level state set to represent the type of ball hitting action and a lower-level state set to represent the stage of ball hitting action in the hierarchical hidden Markov model. In the lower-level state set, the ball-touching anchor point state and the release-from-hand anchor point state are set; The stage states in the lower-level state set are transitioned in a preset order by a preset stage state transition constraint, and the duration of each stage state in the lower-level state set is limited by a preset duration constraint under a unified time base. A pre-established correspondence between the stage states in the lower-level state set and the mode states of the switching linear dynamics system is used to determine the set of mode states allowed for mode state transitions of the switching linear dynamics system. In the non-contact corresponding phase state, restrict the mode state transition from entering the contact corresponding mode state, or limit the transition probability to enter the contact corresponding mode state to below a preset upper limit; In the contact anchor state and the release anchor state, the mode state is allowed to transition between the contact-corresponding mode state and the non-contact-corresponding mode state; when inferring the hierarchical hidden Markov model, the lower-level state is used to limit the set of mode states allowed for the mode state transition of the switching linear dynamic system, and the posterior probability sequence of the mode state is used as observation evidence of the hierarchical hidden Markov model to participate in the state posterior probability update. The iterative update of the switching linear dynamics system and the hierarchical hidden Markov model includes the following steps performed alternately: (a) under the condition of fixing the lower-level state posterior probability or maximum a posteriori path of the hierarchical hidden Markov model, performing switched Kalman filtering or switched Kalman smoothing on the switching linear dynamics system to update the ball state estimation sequence and the pattern state posterior probability sequence; (b) under the condition of fixing the pattern state posterior probability sequence, performing forward-backward inference or maximum a posteriori path decoding on the hierarchical hidden Markov model to update the state posterior probability of the lower-level state, and update the state posterior probabilities of the ball-touching anchor state and the ball-leaving anchor state. (c) Use the update results of steps (a) and (b) as the observation evidence or constraint input for the next iteration; Under the conditions of satisfying the stage state transition constraints and the duration constraints, the switching linear dynamic system and the hierarchical hidden Markov model are iteratively updated until the preset convergence conditions are met. The maximum a posteriori path decoding outputs the ball-hitting action type identification result, the ball-hitting time, the ball-leaving time, and the ball-hitting action phase segmentation results arranged according to a unified time reference. S6. Calculate the preset special indicators based on the segmented results of the hitting action phase, and output the scoring results and error correction prompts.
2. The method for evaluating shot segments based on a Markov model according to claim 1, characterized in that, S1 includes: When acquiring the video data, the acquisition timestamps corresponding to each frame of the video data are recorded, and the frames are sorted according to the acquisition timestamps to generate an image sequence. A unified time base is established based on the collection timestamp, and the collection timestamp is mapped to a time index under the unified time base to generate the time index sequence; When acquiring sensor data while acquiring video data, the sampling timestamp of the sensor data is recorded, and the sampling timestamp is mapped to the unified time base, so that the time index sequence corresponds to the sensor data under the unified time base.
3. The method for evaluating shot segments based on a Markov model according to claim 2, characterized in that, S2 includes: The image sequence is processed frame by frame according to the time index sequence. The human key point detection model is used to output the two-dimensional or three-dimensional coordinates of the human key points at each time index. The two-dimensional or three-dimensional coordinates of the human key points at each time index are arranged according to a unified time reference to generate the human pose sequence. The image sequence is processed frame by frame according to the time index sequence. The sphere target detection model is used to output the two-dimensional or three-dimensional coordinates of the sphere at each time index. The two-dimensional or three-dimensional coordinates of the sphere at each time index are arranged according to a unified time reference to generate the sphere trajectory sequence. When the human posture sequence or the ball trajectory sequence has missing values at some time indices, interpolation is performed to fill in the missing values. When the sampling intervals of the human posture sequence and the ball trajectory sequence are inconsistent, the human posture sequence and the ball trajectory sequence are resampled or interpolated based on the time index sequence, so that the human posture sequence and the ball trajectory sequence correspond one-to-one at each time index, thereby obtaining the time-aligned human posture sequence and the time-aligned ball trajectory sequence.
4. The method for evaluating shot segments based on a Markov model according to claim 1, characterized in that, S3 includes: At each time index, based on the human body key points in the time-aligned human posture sequence and the two-dimensional or three-dimensional coordinates of the ball in the time-aligned ball trajectory sequence, the relative distance between the two-dimensional or three-dimensional coordinates of the ball and the two-dimensional or three-dimensional coordinates of the human body key points corresponding to the hand or forearm is calculated. Based on the two-dimensional or three-dimensional coordinates of the ball at adjacent time indices and the two-dimensional or three-dimensional coordinates of the key points of the human body corresponding to the hand or forearm, calculate the relative velocity between the ball and the hand or forearm. Calculate the velocity or acceleration of the hand or forearm based on the two-dimensional or three-dimensional coordinates of the key points of the human body corresponding to the hand or forearm at adjacent time indices; Calculate the change in velocity or acceleration of the ball's trajectory based on the two-dimensional or three-dimensional coordinates of the ball at adjacent time indices; The relative distance, relative velocity, velocity or acceleration of the hand or forearm, and the change in velocity or acceleration of the ball trajectory are used to form contact candidate features. Before inputting the contact candidate features into the contact probability model, normalization or standardization processing is performed on the contact candidate features. The contact probability model is a probability mapping model based on training parameters or a preset threshold. The contact probability model includes one of the following: logistic regression model, Naive Bayes model, support vector machine probabilistic model, neural network classification model, and threshold-based segmented mapping model. The contact candidate features are mapped to contact probabilities through the contact probability model. The contact probability is a probability value between 0 and 1, and the contact probabilities at each time index are arranged according to a unified time base to generate the contact probability sequence.
5. The ball-hitting segment evaluation method based on Markov model according to claim 1, characterized in that, S4 include: Based on the time-aligned ball trajectory sequence, a switching linear dynamic system is constructed to describe the ball's motion. The switching linear dynamic system consists of ball state, mode state, linear state transition model, and observation model. The ball state includes the ball's position and velocity, and the observation model is used to map the ball state to observations corresponding to the ball trajectory sequence. Set corresponding linear state transition model parameters and process noise parameters for different mode states; The conditional transition probability of the mode state is determined based on the contact probability sequence, such that the transition probability from the mode state corresponding to non-contact to the mode state corresponding to contact increases as the contact probability increases, and the transition probability from the mode state corresponding to contact to the mode state corresponding to non-contact increases as the contact probability decreases. The conditional transition probability is determined by the contact probability at the time index through a monotonic function, which includes a linear truncation function, a sigmoid function, a softmax function, or a piecewise threshold function. The time-varying mode state transition matrix is constructed from the conditional transition probability. Under the conditional transition probability constraint, a switched Kalman filter is performed on the ball trajectory sequence to generate the ball state estimation sequence and the mode state posterior probability sequence. Based on the posterior probability sequence of the mode state, determine the time index at which the dominant mode changes at adjacent time indices, or determine the time index at which the posterior probability of the corresponding mode state exceeds a preset threshold, and arrange the determined time indices according to a unified time base to generate the candidate time sequence for mode switching.
6. The ball-hitting segment evaluation method based on the Markov model according to claim 5, characterized in that, S5 include: A hierarchical hidden Markov model is established based on the time-aligned human posture sequence, the ball state estimation sequence, the contact probability sequence, and the mode switching candidate time sequence. The contact probability sequence and the mode switching candidate time sequence are used to construct observational evidence of the ball-touching anchor state and the release anchor state. The observations of the hierarchical hidden Markov model include human posture features, ball state features, contact probability, and posterior probability of the pattern state. The correspondence includes one of the following: a preset mapping table, a mapping model learned based on training data, or a rule mapping based on a contact probability threshold; The preset convergence condition includes one or a combination of the following: The number of iterations has reached the preset limit; The increment of the log-likelihood value between two consecutive iterations is less than a preset threshold; The time index corresponding to the maximum a posteriori of the ball-touching anchor state or the ball-leaving anchor state remains unchanged during a preset number of iterations.
7. The ball-hitting segment evaluation method based on Markov model according to claim 1, characterized in that, S6 include: Based on the segmentation results of the ball-hitting action phase, the time interval corresponding to each ball-hitting action phase is determined, and the anchor point time point corresponding to the ball-hitting anchor point state and the ball-leaving anchor point state is determined at the time index where the ball-hitting moment and the ball-leaving moment are located. The horizontal position, vertical position, and horizontal reference direction are determined based on a preset coordinate system, which includes one of the following: an image coordinate system, wherein the first axis is the horizontal axis and the second axis is the vertical axis; The site coordinate system established by camera calibration has the vertical axis aligned with the direction of gravity. A coordinate system established based on the direction of gravity determined by inertial sensors; Within the time interval of the ball-throwing action phase, the ball-throwing offset is calculated by the offset of the ball's horizontal position in the ball state estimation sequence relative to the horizontal position of the key points of the human body corresponding to the human torso. At the anchor point time point corresponding to the ball contact anchor point state, the height of the hitting point is calculated based on the vertical position of the ball in the ball state estimation sequence; During the time interval of the hitting action phase corresponding to the ball setting or passing, the forearm platform angle is calculated by the angle between the line connecting the key points of the human body corresponding to the forearm and the horizontal reference direction. Within the time interval of the ball-striking action phase corresponding to the arm-swing acceleration, the peak arm-swing acceleration is calculated based on the maximum value of the acceleration or angular velocity of the key points of the human body corresponding to the hand; the ball-throwing deviation, the ball-striking point height, the forearm platform angle, and the peak arm-swing acceleration are compared with the preset reference range to obtain the deviation of each specific indicator; The scoring result is calculated based on the deviation using a scoring mapping function. The scoring result is a score value or a score level. The scoring mapping function includes one of the following: a linear mapping function, a piecewise mapping function, a sigmoid function, or a weighted summation function. Error correction suggestions are generated based on the specific indicators whose deviation exceeds a preset threshold and their deviation direction. The error correction suggestions are then associated with the ball contact time, the ball release time, or the segmented results of the ball striking action phase and output.
8. The method for evaluating shot segments based on a Markov model according to claim 5, characterized in that, When determining the conditional transition probability of the mode state based on the contact probability sequence, a first threshold for entering the mode state corresponding to contact and a second threshold for exiting the mode state corresponding to contact are set, and the first threshold is greater than the second threshold. In addition, a minimum dwell time constraint is set for the mode state corresponding to contact, so that when the duration of the mode state corresponding to contact is less than the minimum dwell time, the transition to the mode state corresponding to non-contact is not allowed.
9. The method for evaluating shot segments based on a Markov model according to claim 5, characterized in that, The mode states include free flight mode state, hand contact mode state, and forearm contact mode state. Different linear state transition model parameters and process noise parameters are set for the hand contact mode state and the forearm contact mode state, respectively.
10. The method for evaluating shot segments based on a Markov model according to claim 6, characterized in that, The observational evidence for the ball-touching anchor state and the release-from-hand anchor state is obtained by fusing multiple pieces of evidence, including: local maxima of the contact probability sequence, local maxima of the rate of change of ball velocity in the ball state estimation sequence, and local maxima of hand or forearm acceleration in the human posture sequence. The weighted sum or product of the multiple pieces of evidence is used as the observational likelihood of the anchor state to update the posterior probability of the anchor state.
Citation Information
Patent Citations
Motion recognition method and apparatus, terminal device, and sport monitoring system
WO2022057795A1
Ball sports training method and ball sports training system
WO2025119361A1
