Key frame extraction and performance estimation method for standing long jump based on sparse representation
Through the extraction and performance estimation method of standing long jump keyframe based on sparse representation, the problems of low measurement efficiency and insufficient accuracy in the existing technology are solved, and high-precision and reliable long jump distance measurement are achieved, which is suitable for physical education courses and large-scale physical fitness tests .
Patent Information
- Application Number
- CN202411218847.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-09-02
AI Technical Summary
When measuring the standing long jump score, the existing technology has problems such as low measurement efficiency, insufficient accuracy, and susceptible to subjective factors. The video analysis and inertial sensor methods have challenges in equipment and computing resource requirements, making it difficult to teach ordinary sports. Widely used.
Using the method of extracting and estimating the standing long jump keyframes based on sparse representation, the video data is obtained, the motion feature distribution and functional area dictionary is constructed, the candidate keyframe positions are selected, the key action frames are restored using compression perception technology, and combined with Site calibration information calculates the long jump distance.
It realizes high-precision and reliable long-hop measurement, significantly improving measurement efficiency, and is suitable for a large number of measurement scenarios such as physical education courses and large-scale physical fitness tests.
Smart Images

Figure CN118762310B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data recognition, and in particular to a method for extracting key frames and estimating performance of standing long jump based on sparse representation. Background Art
[0002] Standing long jump is an important project widely used in sports testing and sports training. It can effectively evaluate the participants' lower limb explosive power, coordination ability and overall athletic quality. The traditional method of measuring standing long jump results mainly relies on manual visual inspection and tape measurement. This method is simple to operate and requires low equipment, so it is widely used in school sports and mass sports. However, this traditional method has many problems, such as low measurement efficiency, insufficient accuracy, and susceptibility to subjective factors, which seriously affect the accuracy and reliability of the measurement results.
[0003] With the development of science and technology, researchers have begun to explore automatic measurement methods based on video analysis. This type of method usually uses a high-speed camera to record the entire long jump process, and then analyzes the video through image processing and computer vision technology to extract key information and finally calculate the long jump distance. This method has improved the efficiency and accuracy of measurement to a certain extent and reduced human errors. However, methods based on video analysis also face a series of challenges: first, the video sequence of the long jump process is usually long and contains a lot of redundant information, which increases the complexity of processing; second, the athlete's rapid jumping action often causes severe motion blur in the image, affecting the image quality; in addition, the mutual occlusion of body parts during movement also brings difficulties to image analysis.
[0004] In order to overcome these challenges, researchers have proposed a variety of improvement methods. For example, using a multi-view camera system to reduce occlusion problems, using motion prediction algorithms to deal with motion blur, and using deep learning technology to improve the accuracy of image analysis. However, these methods often require complex equipment and a large amount of computing resources, making them difficult to be widely used in ordinary sports teaching and training.
[0005] On the other hand, some researchers have begun to explore measurement methods based on inertial sensors. This method fixes small inertial sensors on athletes to record acceleration and angular velocity data during exercise, and then calculates the long jump distance through algorithms. This method avoids some problems in video analysis, but it also brings new challenges, such as the way sensors are fixed, data synchronization and fusion, etc.
[0006] In addition, some researchers have tried to combine computer vision technology with machine learning methods to build a model of long jump movements through training with a large amount of data, thereby improving the accuracy of measurement. Although this method performs well in some scenarios, it has high requirements on the quality and quantity of training data, and its generalization ability needs to be verified. Summary of the invention
[0007] Based on this, it is necessary to provide a standing long jump key frame extraction and performance estimation method based on sparse representation that can improve efficiency in response to the above technical problems.
[0008] A method for extracting key frames and estimating performance of standing long jump based on sparse representation comprises the following steps:
[0009] Acquire first calibration long jump video data of a standing long jumper, and determine a motion feature distribution corresponding to the calibration long jump video data;
[0010] The calibration long jump video data is pre-classified using the motion feature distribution to obtain a pre-classification result, and the test site is divided into multiple functional areas according to the pre-classification result; the pre-classification result at least includes a short-distance high arc type, a long-distance low arc type and a medium-distance balanced type;
[0011] Based on the functional area, obtaining second calibration long jump video data, and constructing an overcomplete dictionary of standing long jumpers according to the second calibration long jump video data; the overcomplete dictionary includes typical action frames corresponding to the functional area;
[0012] Collecting measurement standing long jump video data, and sparsely encoding the measurement standing long jump video data based on an overcomplete dictionary to obtain sparse representation coefficients;
[0013] Selecting multiple candidate key frame positions in the standing long jump measurement video data according to the sparse representation coefficient;
[0014] For the candidate key frame positions, the reconstruction error and sparsity corresponding to the candidate key frame positions are evaluated;
[0015] When the relationship between the reconstruction error and the sparsity satisfies a preset condition, determining the optimal key frame position from the candidate key frame positions based on the sparse representation coefficient, the adjusted reconstruction error and the sparsity;
[0016] Based on the optimal key frame position, the compressed sensing technology is used to recover the key action frames of the long jump process from the sparse representation coefficients. The key action frames include the take-off frame and the landing frame.
[0017] Locate the key points of the take-off frame and the landing frame, and calculate the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired field calibration information;
[0018] The long jump distance of the standing long jumper is calculated based on the spatial coordinates of the take-off point and the landing point of the standing long jumper.
[0019] In one embodiment, the step of obtaining calibrated long jump video data of a standing long jumper and determining the distribution of motion features corresponding to the calibrated long jump video data comprises the steps of:
[0020] Preprocessing the calibration long jump video data to obtain single calibration long jump video sub-data; the calibration long jump video sub-data includes a complete long jump action of a standing long jumper;
[0021] Extract key motion features from the calibrated long jump video sub-data; motion features include take-off angle, take-off speed, aerial posture, landing technique and jump distance;
[0022] Use image processing technology to process motion features and quantify them;
[0023] Perform statistical analysis on the quantified motion features to obtain the motion feature distribution.
[0024] In one embodiment, the steps of pre-classifying the calibration long jump video data using the motion feature distribution to obtain the pre-classification result, and dividing the test site into a plurality of functional areas according to the pre-classification result include the steps of:
[0025] Calculate the mean and standard deviation of the motion features of the calibration long jump video data, and obtain the coefficient of variation based on the mean and standard deviation;
[0026] Normalizing the maximum and minimum values of the motion features of the calibration long jump video data to obtain a normalized result;
[0027] Based on the coefficient of variation, normalized results and motion feature distribution, the aggregation energy is obtained through experiments and cross-validation;
[0028] A pre-classification result is obtained according to the aggregated energy, and the test site is divided into multiple functional areas according to the pre-classification result.
[0029] In one embodiment, the step of acquiring second calibrated long jump video data based on the functional area and constructing an overcomplete dictionary of standing long jumpers according to the second calibrated long jump video data comprises the steps of:
[0030] According to the functional area, the data acquisition strategy is optimized to obtain the second calibration long jump video data;
[0031] Preprocessing the second calibration long jump video data to adjust pixels of all frames of the second calibration long jump video data to MxN pixels; the preprocessing includes image denoising, contrast adjustment and size standardization;
[0032] The feature vector is extracted from the preprocessed second calibration long jump video data using the histogram of oriented gradients feature descriptor, and the dimension reduction of the feature vector is performed using the principal component analysis algorithm;
[0033] The K-means clustering algorithm is used to process the feature vector after dimension reduction and construct the initial dictionary;
[0034] The K singular value decomposition algorithm is used to optimize and refine the initial dictionary to obtain an overcomplete dictionary.
[0035] In one embodiment, the step of collecting the measured standing long jump video data and sparsely encoding the measured standing long jump video data based on an overcomplete dictionary to obtain sparse representation coefficients includes the steps of:
[0036] Preprocessing the video data of standing long jump measurement;
[0037] The feature vector is extracted from the preprocessed measured standing long jump video data using the histogram of oriented gradients feature descriptor;
[0038] The principal component analysis algorithm is used to reduce the dimension of the feature vector to obtain a feature vector with the same dimension as the overcomplete dictionary.
[0039] The orthogonal matching pursuit algorithm is used to optimize the feature vector after dimension reduction to obtain the sparse representation coefficient.
[0040] In one embodiment, the step of selecting a plurality of candidate key frame positions for measuring standing long jump video data according to the sparse representation coefficient comprises the steps of:
[0041] Calculate the sparsity of each frame in the standing long jump video data, and calculate the first-order difference of the sparsity;
[0042] Calculate the energy of each frame in the standing long jump video data and calculate the first-order difference of the energy;
[0043] Calculate and measure the atomic activity of standing long jump video data in time series;
[0044] Calculating the cosine similarity of sparse representation coefficients of adjacent frames in the standing long jump video data;
[0045] A comprehensive scoring function is constructed based on the first-order difference of sparsity, the first-order difference of energy, atomic activity and cosine similarity;
[0046] Based on the comprehensive score function, a peak detection algorithm is used to determine the candidate key frame positions in the standing long jump measurement video data.
[0047] In one embodiment, the step of evaluating the reconstruction error and sparsity corresponding to the candidate key frame position further includes the following steps:
[0048] For the candidate key frame positions, evaluating the preliminary reconstruction error and preliminary sparsity of the candidate key frame positions;
[0049] Normalizing the initial reconstruction error to obtain a normalized reconstruction error value;
[0050] Normalize the initial sparsity to obtain a normalized sparsity value;
[0051] The reconstruction error normalization value and the sparsity normalization value are processed based on the weight coefficient of the frame type to obtain the reconstruction error and sparsity.
[0052] In one embodiment, based on the optimal key frame position, the key action frame of the long jump process is sparsely recovered from the representation coefficient using the compressed sensing technology, and the key action frame includes the step of taking-off frame and landing frame, including the steps of:
[0053] Based on the sparse representation coefficients and the overcomplete dictionary, the feature vectors of the optimal key frame position at each moment are reconstructed through matrix multiplication;
[0054] The reconstructed feature vector is converted into a reconstructed gradient field, and the reconstructed gradient is converted into a reconstructed image sequence by solving the Poisson equation;
[0055] In the reconstructed image sequence, a first image subsequence of 2 frames before and after the estimated take-off moment and a second image subsequence of 2 frames before and after the estimated landing moment are selected;
[0056] Using sub-frame interpolation technology to improve the temporal resolution of the first image subsequence and the second image subsequence;
[0057] The first image subsequence and the second image subsequence with improved time resolution are processed using a motion change detection function to obtain a first motion change intensity sequence corresponding to the first image subsequence and a second motion change intensity sequence corresponding to the second image subsequence, respectively;
[0058] The first action change intensity sequence is processed by an adaptive threshold method to obtain a preliminary take-off frame, and the second action change intensity sequence is processed by an adaptive threshold method to obtain a preliminary landing frame;
[0059] The preliminary take-off frame is processed by a quadratic interpolation method, and the take-off frame is extracted from the processing result. The preliminary landing frame is processed by a quadratic interpolation method, and the landing frame is extracted from the processing result.
[0060] In one embodiment, the step of locating the key points of the take-off frame and the landing frame, and calculating the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired field calibration information, includes the steps of:
[0061] The mapping relationship between the image coordinate system and the space coordinate system is established using the site calibration information;
[0062] Calculate the take-off image coordinates of the key points of the take-off frame, and calculate the landing image coordinates of the key points of the landing frame;
[0063] Based on the mapping relationship, the take-off image coordinates are converted into the spatial coordinates of the take-off point, and the landing image coordinates are converted into the spatial coordinates of the landing point.
[0064] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0065] One of the above technical solutions has the following advantages and beneficial effects:
[0066] The present invention provides a method for extracting key frames and estimating performance of standing long jump based on sparse representation, and comprises the following steps: obtaining first calibrated long jump video data of a standing long jumper, and determining the motion feature distribution corresponding to the calibrated long jump video data; pre-classifying the calibrated long jump video data using the motion feature distribution to obtain a pre-classification result, and dividing the test site into multiple functional areas according to the pre-classification result; obtaining second calibrated long jump video data based on the functional areas, and constructing an overcomplete dictionary of the standing long jumper according to the second calibrated long jump video data; collecting measured standing long jump video data, and sparsely encoding the measured standing long jump video data based on the overcomplete dictionary to obtain a sparse representation coefficient; selecting multiple candidate key frames in the measured standing long jump video data according to the sparse representation coefficient. Frame position; for the candidate key frame position, evaluate the reconstruction error and sparsity corresponding to the candidate key frame position; when the relationship between the reconstruction error and the sparsity meets the preset conditions, based on the sparse representation coefficient, the adjusted reconstruction error and sparsity, determine the optimal key frame position from the candidate key frame position; based on the optimal key frame position, use compressed sensing technology to recover the key action frame of the long jump process from the sparse representation coefficient, and the key action frame includes the take-off frame and the landing frame; locate the key points of the take-off frame and the landing frame, and calculate the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired site calibration information; based on the spatial coordinates of the take-off point and the landing point of the standing long jumper, measure the long jump distance of the standing long jumper. This application ensures the high accuracy and reliability of the measurement results through accurate three-dimensional coordinate calculation and multiple verification steps. The entire measurement process can be completed automatically within a few seconds, which significantly improves the measurement efficiency, and is particularly suitable for scenarios where a large number of measurements are required, such as physical education courses or large-scale physical fitness tests. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 It is a flowchart of the method for extracting key frames and estimating performance of standing long jump based on sparse representation in an embodiment of the present application.
[0068] Figure 2 Schematic diagram of the internal structure of a computer device in an embodiment of the present application. DETAILED DESCRIPTION
[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0070] In order to achieve the above object, in one embodiment, Figure 1 As shown, a method for extracting key frames and estimating performance of standing long jump based on sparse representation is provided, comprising the following steps:
[0071] Step S110, obtaining first calibrated long jump video data of a standing long jumper, and determining a motion feature distribution corresponding to the calibrated long jump video data.
[0072] First, data collection is carried out in physical education classes or physical tests in schools. A high-definition camera is fixed on the side of the long jump area to ensure that the entire long jump process of the students can be fully captured, and the entire process from preparation to landing is recorded. In order to obtain a representative data sample, long jump videos of students of different grades and genders are collected. Usually, data from 100 to 200 students are collected, and each student performs 3 valid jumps, which can ensure the diversity and statistical significance of the data. These data are compiled into the first calibration long jump video data.
[0073] After the first calibration long jump video data is collected, necessary processing is performed. In one example, the step of obtaining the calibration long jump video data of a standing long jumper and determining the distribution of motion features corresponding to the calibration long jump video data includes the following steps: preprocessing the calibration long jump video data to obtain single calibration long jump video sub-data; the calibration long jump video sub-data includes a complete long jump action of a standing long jumper; extracting key motion features from the calibration long jump video sub-data; the motion features include take-off angle, take-off speed, aerial posture, landing skills and jumping distance; using image processing technology to process the motion features and quantify the motion features; performing statistical analysis on the quantified motion features to obtain the distribution of motion features.
[0074] Specifically, the long video clips shot continuously are divided into single long jump segments, each of which contains a student's complete long jump action (i.e., calibrated long jump video sub-data). Then, basic quality checks are performed on these video clips to remove those that are blurry due to improper shooting angles, insufficient lighting, or other reasons. This step ensures the data quality for subsequent analysis, and all retained videos can clearly show the students' long jump actions.
[0075] Next, we extract key motion features from these preprocessed videos. Considering the characteristics of standing long jump and the characteristics of the student population, we focus on the following features: take-off angle, take-off speed, aerial posture, landing skills, and jump distance.
[0076] The take-off angle is determined by analyzing the angle between the student's body and the ground at the moment of take-off. The take-off speed is calculated from the student's speed from the beginning of the jump to the moment of leaving the ground. The aerial posture mainly observes the student's body posture in the air, especially the change in the angle of the torso. The landing technique analyzes the student's cushioning action and balance control when landing. The jump distance measures the distance from the take-off line to the closest landing point after the student lands.
[0077] To quantify these features, a range of image processing techniques are used. For example, the take-off angle and speed are estimated by tracking the position of the student's center of gravity. The key body parts of the student, such as the head, torso center, hips, etc., are marked in the video, and the movement of the center of gravity is calculated through the movement trajectory of these points. For the analysis of aerial posture, an edge detection algorithm is used to extract the outline of the student's body in each frame, and then the changes of this outline during the flight in the air are analyzed. The analysis of landing skills mainly relies on the image sequence at the moment of landing and the short time afterwards, focusing on the degree of leg bending and body stability.
[0078] After extracting these features, a detailed statistical analysis is performed to determine the distribution of the motion features. For each extracted feature, its mean, standard deviation, maximum, and minimum values are calculated. These statistics provide an overall overview of the student population's performance in various aspects of the standing long jump. For example, it may be found that the mean value of the take-off angle is 40 degrees and the standard deviation is 5 degrees; the mean value of the take-off speed is 2.5 meters per second and the standard deviation is 0.3 meters per second; and the mean value of the jump distance is 1.8 meters and the standard deviation is 0.2 meters. These data reflect the general performance level and range of students in the standing long jump.
[0079] Through this series of data processing and analysis, the motion characteristic distribution of the initial long jump data was finally obtained.
[0080] Step S120, pre-classify the calibration long jump video data using the motion feature distribution to obtain a pre-classification result, and divide the test site into multiple functional areas according to the pre-classification result, wherein the pre-classification result at least includes a short-distance high arc type, a long-distance low arc type, and a medium-distance balanced type.
[0081] The first calibration long jump video data is pre-classified using clustering algorithms in machine learning, such as K-means or hierarchical clustering. These algorithms can automatically identify patterns in the data and group similar long jump movements together.
[0082] In one example, the steps of pre-classifying the calibrated long jump video data using the motion feature distribution to obtain the pre-classification results, and dividing the test site into multiple functional areas according to the pre-classification results include the following steps: calculating the mean and standard deviation of the motion features of the calibrated long jump video data, and obtaining the coefficient of variation according to the mean and standard deviation; performing feature normalization on the maximum and minimum values of the motion features of the calibrated long jump video data to obtain the normalized results; obtaining the aggregated energy through experiments and cross-validation based on the coefficient of variation, the normalized results and the motion feature distribution; obtaining the pre-classification results according to the aggregated energy, and dividing the test site into multiple functional areas according to the pre-classification results.
[0083] Specifically, 1. Feature selection: Based on the mean and standard deviation of each motion feature calculated in step S110, the feature with a larger coefficient of variation (standard deviation divided by the mean) is selected as the main basis for clustering. For example, if the coefficient of variation of the take-off angle is large, while the coefficient of variation of the take-off speed is small, the take-off angle feature may be given more importance during clustering.
[0084] 2. Feature normalization: Use the maximum and minimum values of each motion feature obtained in step S110 to perform feature normalization to ensure that features of different dimensions can be fairly compared during the clustering process.
[0085] 3. Determine the number of clusters: Through experiments and cross-validation, combined with the feature distribution obtained in step S110, determine the most suitable number of clusters. Usually between 3 and 5, this number can reflect the main long jump pattern differences without making subsequent analysis too complicated. For example, if some features show obvious multi-modal distribution, you may tend to choose more clusters.
[0086] The pre-classification results may include several categories such as "short distance high arc type", "long distance low arc type" and "medium distance balanced type". The feature descriptions of these categories are directly based on the statistical data obtained in step S110: Short distance high arc type: the take-off angle is greater than the average value (40 degrees) plus one standard deviation (5 degrees), that is, greater than 45 degrees; the take-off speed is less than the average value (2.5 meters / second) minus one standard deviation (0.3 meters / second), that is, less than 2.2 meters / second. Long distance low arc type: the take-off angle is less than the average value minus one standard deviation, that is, less than 35 degrees; the take-off speed is greater than the average value plus one standard deviation, that is, greater than 2.8 meters / second. Medium distance balanced type: the take-off angle and take-off speed are both within the range of plus or minus one standard deviation from the average value.
[0087] According to the pre-classification results, the test site is divided into multiple functional areas. The purpose of this division is to create the most suitable data collection environment for different types of long jump actions to ensure the accuracy of the measurement. When dividing the site, the main considerations are the layout of the camera, the division of the measurement area, the setting of the data collection points, and the application of the optical marking system.
[0088] In terms of camera arrangement, high-speed cameras are set at different locations of the venue according to the characteristics of different long jump modes. For example, for short-distance high-arc long jump, based on its larger take-off angle (>45 degrees), multiple cameras are arranged near the take-off area and in the aerial action area to capture vertical movement. The elevation angle of the camera is set based on the maximum take-off angle obtained in step S110. For long-distance low-arc long jump, considering its larger take-off speed (>2.8 meters / second), it is necessary to set the camera at a farther distance to ensure that the entire jump process can be fully captured, especially the longer horizontal motion trajectory. The field of view of the camera is set based on the maximum jump distance obtained in step 110 (the average value is 1.8 meters plus two standard deviations of 0.4 meters, that is, about 2.6 meters).
[0089] Division of measurement area: For categories that are expected to jump farther (such as long-distance low-arc type), the measurement area will be extended to the average jump distance (1.8 meters) plus three standard deviations (0.6 meters), that is, about 2.4 meters, to ensure that all possible landing points can be accurately recorded. At the same time, for categories that jump closer (such as short-distance high-arc type), the measurement accuracy of the close-range area will be increased, and higher-resolution measurement equipment or more dense marking points may be used. The spacing between marking points can be set to 1 / 10 of the minimum jump distance obtained in step S110.
[0090] Data collection points are set up at different locations in the venue in order to collect the most relevant data for different categories of long jump patterns. For example, for short-distance high-arc long jumps, based on their large take-off angle variation (standard deviation 5 degrees), more cameras may be added near the take-off area to accurately capture the changes in take-off angle and initial velocity. For long-distance low-arc long jumps, considering their large horizontal velocity variation (standard deviation 0.3 m / s), cameras may be evenly distributed throughout the jump path to obtain complete velocity variation data.
[0091] Application of optical marker system: Depending on the characteristics of different long jump modes, optical markers may be placed at different locations on the student. These markers help to more accurately track the movement of key body parts in subsequent image processing. For example, for long jump modes that require more attention to upper body posture (such as those with large changes in posture in the air), more marker points may be added to the shoulders and arms, and the density of the marker points can be determined based on the amplitude of the posture change obtained in step S110.
[0092] Through such functional area division, the most suitable data collection environment can be created for each long jump mode, thereby improving the accuracy and efficiency of the overall measurement.
[0093] Step S130 , based on the functional area, obtaining the second calibrated long jump video data, and constructing an overcomplete dictionary of standing long jumpers according to the second calibrated long jump video data; the overcomplete dictionary contains typical action frames corresponding to the functional area.
[0094] In one example, based on the functional area, the second calibrated long jump video data is obtained, and an over-complete dictionary of standing long jumpers is constructed according to the second calibrated long jump video data, including the steps of: optimizing the data acquisition strategy according to the functional area to obtain the second calibrated long jump video data; preprocessing the second calibrated long jump video data to adjust the pixels of all frames of the second calibrated long jump video data to MxN pixels; the preprocessing includes image denoising, contrast adjustment and size standardization; using a directional gradient histogram feature descriptor to extract a feature vector from the preprocessed second calibrated long jump video data, and using a principal component analysis algorithm to reduce the dimension of the feature vector; using a K-means clustering algorithm to process the reduced feature vector to construct an initial dictionary; using a K-singular value decomposition algorithm to optimize and refine the initial dictionary to obtain an over-complete dictionary.
[0095] Specifically, first, according to the functional areas divided in step S120 (such as "short-distance high arc type", "long-distance low arc type" and "medium-distance balanced type", etc.), the data acquisition strategy is optimized to re-acquire the second calibration long jump video data. For each functional area, the position, angle and parameter settings of the camera are adjusted. For example, for the high-arc type area, the coverage of the vertical viewing angle is increased; for the long-distance type area, the horizontal field of view is expanded. Sampling is performed at a fixed time interval of 0.1 seconds, but the sampling frequency is appropriately increased during key action stages (such as take-off and landing moments). This differentiated acquisition strategy ensures that representative samples of various long jump modes are captured while avoiding redundant data.
[0096] Then, each frame in the second calibration long jump video data is preprocessed, including image denoising, contrast adjustment and size normalization, and all frames are uniformly adjusted to MxN pixels (e.g., 256x256 pixels). Image denoising uses the non-local means algorithm, which can effectively remove noise while preserving image details. Contrast adjustment uses adaptive histogram equalization (CLAHE), which can enhance local contrast while avoiding over-enhancing noise. Size normalization uses bicubic interpolation, which can effectively handle image scaling while maintaining image quality. According to the characteristics of different functional areas, the parameters of these preprocessing steps can be fine-tuned to optimize the image quality of each long jump mode.
[0097] Next, feature extraction is performed using the Histogram of Directed Gradients (HOG) feature descriptor. HOG features have good illumination invariance and geometric invariance, and are particularly suitable for describing human movements. The specific process is to divide the 256x256 image into 32x32 cells of 8x8 pixels, calculate the gradient direction histogram for each cell, use 9 direction intervals (0°-180°, one interval every 20°), and then perform L2-Norm normalization on the 3x3 cell block. This process obtains a feature vector of length 3780 ((32-2)x(32-2)x9x4=3780). According to the characteristics of different functional areas, the HOG parameters can be appropriately adjusted. For example, for areas where the take-off angle needs to be accurately captured, the resolution of the gradient direction can be increased.
[0098] Considering the computational complexity, the principal component analysis (PCA) algorithm can be optionally applied to reduce the dimensionality. PCA projects the original features into a new feature space through linear transformation. The new features are mutually orthogonal and sorted by variance. Retaining the principal components that explain 95% of the variance can usually reduce the feature dimension to about 250. This dimensionality reduction method can not only reduce the amount of calculation, but also remove noise and improve the expressiveness of the features. The application of PCA also takes into account the characteristics of different functional areas and retains the most representative principal components for each area.
[0099] With the preprocessed feature vectors, we start to build the initial dictionary. We use the K-means clustering algorithm to cluster all the reduced feature vectors into 500 categories, with the center of each category as an initial dictionary atom. The advantage of the K-means algorithm is that it is simple and efficient, and can quickly obtain an initial dictionary. 500 categories are selected to ensure the richness of the dictionary without excessive calculation. In this process, considering the characteristics of different functional areas, different numbers of initial atoms can be assigned to each area to reflect the complexity and importance of each area.
[0100] The optimization and refinement of the dictionary is done by the K-SVD algorithm. The K-SVD algorithm is an iterative algorithm, and each iteration includes two main steps: sparse coding and dictionary update. In the sparse coding step, the overcomplete dictionary D is fixed, and the optimization problem is solved for each training sample x: minimize ||α||0, so that . Here, ||·||0 represents the L0 norm, that is, the number of non-zero elements, and ε is the maximum allowed reconstruction error, set to 1% of the original feature variance. α is a sparse coefficient vector, which represents how to represent the signal x using a linear combination of atoms in the dictionary D. D is an overcomplete dictionary, whose column vectors are basic "atomic" signals. Dα is the matrix product of the overcomplete dictionary D and the sparse coefficient α, representing the reconstructed signal. This problem is solved by the orthogonal matching pursuit (OMP) algorithm. The core idea of the OMP algorithm is to select the atom most relevant to the current residual in each iteration, and then update the support set and residual.
[0101] In the overcomplete dictionary update step, the sparse coefficients are fixed and each atom in the overcomplete dictionary is updated. For the kth atom, solve: minimize ||E_k-d_k·α_k^T||_F. Where E_k is the error matrix after removing the contribution of other atoms, d_k is the atom to be updated, α_k^T is the corresponding sparse coefficient row, and ||·||_F represents the Frobenius norm. This problem is solved by singular value decomposition (SVD). Specifically, SVD decomposition is performed on E_k, and the left singular vector corresponding to the maximum singular value is taken as the new d_k, and the maximum singular value multiplied by the right singular vector is taken as the new α_k^T. Repeating these two steps 100 times usually results in a high-quality overcomplete dictionary. In this optimization process, the sparse constraint parameters can be adjusted according to the importance of different functional areas. For example, more overcomplete dictionary atoms are allowed to be used to represent key action frames (such as take-off and landing moments).
[0102] In order to improve the representation ability of the overcomplete dictionary, multi-scale features are introduced. The above process is repeated at the 1 / 2 and 1 / 4 scales of the original image to obtain overcomplete dictionaries of three scales, which are then combined into the final overcomplete dictionary. This multi-scale method can capture action features at different scales and improve the representation ability of the overcomplete dictionary. The number of atoms in the final overcomplete dictionary is set to 1.5 times the feature dimension, that is, 375 atoms, to ensure the overcompleteness of the overcomplete dictionary.
[0103] During the construction process, data augmentation is performed on the original images, including ±5° rotation, ±5% scaling, and horizontal flipping. These transformed images are also used in the overcomplete dictionary learning process. The method and degree of data augmentation can be adjusted according to the characteristics of different functional areas to improve the adaptability of the overcomplete dictionary to various changes.
[0104] Finally, the overcomplete dictionary is fine-tuned. Atoms with a usage frequency of less than 1% are deleted. This frequency is calculated by dividing the number of times used in the validation set by the total number of samples. This step can remove redundant or invalid atoms and improve the efficiency of the overcomplete dictionary. At the same time, atoms specifically for key actions are added to each functional area, including the take-off moment, the highest point in the air, and the landing moment. These specific atoms are generated by the average of 50 typical samples in each functional area to ensure that the overcomplete dictionary can accurately represent the key moments of various long jump modes.
[0105] Step S140 , collecting the measured standing long jump video data, and sparsely encoding the measured standing long jump video data based on an overcomplete dictionary to obtain sparse representation coefficients.
[0106] Preprocess the newly acquired measured standing long jump video data. This process is similar to the preprocessing in S130, including frame extraction, denoising, contrast adjustment, and size normalization. Video frames are sampled at fixed time intervals to ensure that continuous changes throughout the long jump process are captured. For example, for a typical long jump action, a certain number of key frames will be extracted for processing. Each extracted frame is resized to a specific pixel size, consistent with the image size used when building the dictionary.
[0107] In one example, measured standing long jump video data is collected, and sparse encoding is performed on the measured standing long jump video data based on an overcomplete dictionary to obtain a sparse representation coefficient, including the steps of: preprocessing the measured standing long jump video data; using a directional gradient histogram feature descriptor to extract a feature vector from the preprocessed measured standing long jump video data; using a principal component analysis algorithm to reduce the dimension of the feature vector to obtain a reduced dimension feature vector with the same dimension as the overcomplete dictionary; and using an orthogonal matching pursuit algorithm to optimize the reduced dimension feature vector to obtain a sparse representation coefficient.
[0108] The preprocessing method in this step is the same as step S130 and will not be repeated here. Feature vectors are extracted for each frame after preprocessing. The same directional gradient histogram (HOG) feature descriptor as that used in the overcomplete dictionary construction is used. Specifically, the image is divided into cells of several pixels, the gradient direction histogram is calculated for each cell, a certain number of direction intervals are used, and then L2-Norm normalization is performed on the cell blocks of a specific size. This process obtains a feature vector of a specific length. The calculation of the feature vector length takes into account the following factors:
[0109] The image is segmented into a certain number of cells.
[0110] When computing HOG features, blocks of cells of a certain size need to be considered, which means that cells at the edges may not form a complete block.
[0111] Each cell block contains gradient information in a specific number of directions.
[0112] The feature vectors are computed over overlapping blocks.
[0113] In order to maintain consistency with the dictionary, the principal component analysis (PCA) transformation matrix obtained in the overcomplete dictionary construction phase is optionally used to reduce the feature vector to the same dimension as when the overcomplete dictionary was constructed. The PCA transformation not only reduces the dimension, but also removes noise and improves the expressiveness of the features. The transformation process can be expressed as: x'=W^T*x, where x is the original feature vector, W is the PCA transformation matrix, and x' is the feature vector after dimensionality reduction.
[0114] Now, for each frame in the standing long jump video data, we have a feature vector that matches the dimensions of the overcomplete dictionary atoms. The goal is to represent this feature vector using a linear combination of the atoms in the overcomplete dictionary, while requiring the use of as few atoms as possible. Specifically, for each frame’s feature vector x, we need to solve the following optimization problem:
[0115] Minimize ||α||0
[0116] Make
[0117] Here, x is the feature vector of the current frame, D is the overcomplete dictionary constructed in step S130, α is the sparse coefficient vector to be solved, ||·||0 represents the L0 norm, that is, the number of non-zero elements, and ε is the allowed reconstruction error threshold. ε is set to a certain proportion of the variance of the original feature vector, which strikes a good balance between ensuring reconstruction quality and computational efficiency.
[0118] The orthogonal matching pursuit (OMP) algorithm is used to solve this optimization problem. The basic idea of the OMP algorithm is to iteratively select the dictionary atom that can best reduce the residual, and then update the coefficients and residual. The specific steps are as follows:
[0119] 1. Initialization: Set the residual r0=x, the sparse coefficient α=0, and the support set .
[0120] 2. Atom selection: In each iteration, find the dictionary atom that is most relevant to the current residual:
[0121] j=argmax|<r_i,d_j> |, where d_j is the j-th column of dictionary D.
[0122] Computing the inner product can be accelerated using the Fast Fourier Transform (FFT), especially when the dictionary size is large.
[0123] 3. Support set update: S = S ∪ {j}.
[0124] 4. Least squares solution: Solve .
[0125] S is the support set, which represents the index set of the selected dictionary atoms.
[0126] D_S is the submatrix in dictionary D that contains only the columns corresponding to the indices in support set S.
[0127] α_S is the non-zero element in α corresponding to the support set S.
[0128] 5. Residual update: r_i+1=x-D_Sα_S.
[0129] 6. Termination judgment: If Or if the preset maximum number of non-zero coefficients is reached, stop; otherwise, return to step 2.
[0130] Through this process, the sparse representation coefficient α of each frame is obtained. Most elements of this coefficient vector are zero, and only a few elements are non-zero, which correspond to the dictionary atoms used to reconstruct the current frame.
[0131] Applying this process to the entire video sequence results in a series of sparse representation coefficients {α_t|t=1, 2, ..., T}, where T is the total number of frames in the video sequence. These sparse representation coefficients form a compact and information-rich representation of the original video sequence. Each α_t not only encodes the visual content of the current frame, but also implicitly contains the temporal information of the action.
[0132] This sparse representation method greatly reduces the dimensionality of the data, making subsequent processing more efficient. Due to the use of an overcomplete dictionary designed specifically for standing long jump, this representation can effectively capture the key features of the long jump action. At the same time, sparsity makes it easy to identify the most important visual features in each frame.
[0133] Step S150 , selecting a plurality of candidate key frame positions in the standing long jump measurement video data according to the sparse representation coefficients.
[0134] In one example, the steps of selecting multiple candidate key frame positions in the measured standing long jump video data based on the sparse representation coefficient include the steps of: calculating the sparsity of each frame in the measured standing long jump video data, and calculating the sparsity first-order difference of the sparsity; calculating the energy of each frame in the measured standing long jump video data, and calculating the energy first-order difference of the energy; calculating the atomic activity of the measured standing long jump video data in time series; calculating the cosine similarity of the sparse representation coefficients of adjacent frames in the measured standing long jump video data; constructing a comprehensive scoring function based on the first-order difference of sparsity, the first-order difference of energy, the atomic activity and the cosine similarity; based on the comprehensive scoring function, using a peak detection algorithm to determine the candidate key frame positions in the measured standing long jump video data.
[0135] Specifically, the sparse representation coefficients are analyzed to identify key moments in the standing long jump video data. This process mainly considers the change pattern of the sparse representation coefficients in the time series, assuming that when the long jump action changes significantly, the dictionary atom combination used to represent the frame will also change accordingly, which will be reflected in the change of the sparse representation coefficients.
[0136] Calculate the sparsity of each frame of the standing long jump video data, that is, the number of non-zero coefficients, defined as S(t)=||α_t||0. Sudden changes in sparsity usually indicate key turning points in the action. For example, at the moment of take-off, more dictionary atoms may be needed to describe the complex body posture, resulting in increased sparsity. The sparsity can be calculated by the following steps:
[0137] 1. For each time point t, get the corresponding sparse coefficient vector α_t.
[0138] 2. Set a small threshold ε (e.g. 1e-6) to determine whether the coefficient is zero.
[0139] 3. Traverse each element in α_t and count the number of elements whose absolute value is greater than ε.
[0140] 4. The number of non-zero elements obtained by counting is taken as the value of S(t).
[0141] At the same time, the "energy" of each frame is calculated, defined as the L2 norm of the sparse representation coefficients: The change in energy reflects the change in the intensity of the action. Usually, the energy value reaches its peak value at the most intense moment of the action (such as take-off or landing). The specific steps for calculating energy are as follows:
[0142] 1. For each time point t, get the corresponding sparse coefficient vector α_t.
[0143] 2. Calculate the square of each element in α_t.
[0144] 3. Sum all the squared values.
[0145] 4. Take the square root of the sum to get the value of E(t).
[0146] Optionally, first-order differences are calculated for both metrics, i.e., ΔS(t)=S(t+1)-S(t) and ΔE(t)=E(t+1)-E(t). Larger difference values may indicate the location of a keyframe. The steps for calculating the difference are as follows:
[0147] 1. For time point t, obtain S(t) and S(t+1) (or E(t) and E(t+1)).
[0148] 2. Calculate the difference between two adjacent time points.
[0149] 3. Take the difference as the value of ΔS(t) (or ΔE(t)).
[0150] Furthermore, a more fine-grained analysis is introduced: tracking the activity of each dictionary atom in the sequence. Define the atom activity function A_j(t)=|α_t[j]|, where α_t[j] is the jth element of α_t. These time series provide rich information. Special attention is paid to atoms related to key actions (such as take-off, aerial posture, and landing). The activity peaks of these atoms often correspond to key frames. The steps to calculate the activity of atoms are as follows:
[0151] 1. For each time point t and dictionary atom index j, get the value of α_t[j].
[0152] 2. Calculate the absolute value of α_t[j].
[0153] 3. The calculated absolute value is used as the value of A_j(t).
[0154] In order to capture the continuity of the action, it is preferred to calculate the cosine similarity of the sparse representation coefficients of adjacent frames: C(t)=cosine(α_t, α_{t+1}). A sharp drop in C(t) may indicate a significant change in the state of the action and is a potential key frame position. The steps for calculating cosine similarity are as follows:
[0155] 1. For time point t, obtain α_t and α_{t+1}.
[0156] 2. Calculate the dot product of α_t and α_{t+1}.
[0157] 3. Calculate the L2 norm of α_t and α_{t+1} respectively.
[0158] 4. Divide the dot product by the product of the two L2 norms to get the value of C(t).
[0159] Combining these analysis results, a comprehensive score function is constructed:
[0160] Score(t)=w1·|ΔS(t)|+w2·|ΔE(t)|+w3·max_j(A_j(t))+w4·(1-C(t))
[0161] Where w1, w2, w3, w4 are weight coefficients.
[0162] After obtaining the comprehensive score function, the peak detection algorithm is used to determine the candidate key frame positions. The sliding window method is used, and the window size is set to 1 / 5 of the video frame rate (about 0.2 seconds). In each window, the local maximum is found. In order to avoid detecting too many candidate key frame positions, an adaptive threshold θ is set. This threshold is defined as the average value of Score(t) plus 1.5 times the standard deviation. Only the local maximum point with a score exceeding θ is retained as a candidate key frame position. The specific steps of peak detection are as follows:
[0163] 1. Determine the sliding window size W, for example, set it to 1 / 5 of the video frame rate.
[0164] 2. For each time point t, find the local maximum of Score(t) in the window of [tW / 2, t+W / 2].
[0165] 3. Calculate the mean μ and standard deviation σ of the entire sequence Score(t).
[0166] 4. Set the threshold θ = μ + 1.5σ.
[0167] 5. If a local maximum is greater than θ, the corresponding time point is marked as a candidate key frame position.
[0168] Considering the characteristics of the standing long jump action, additional timing constraints are optionally imposed. The interval between adjacent candidate keyframe positions is required to be no less than 0.3 seconds. This constraint is based on the principle of human kinematics and ensures that the real action changes are captured, rather than instantaneous fluctuations or measurement noise. The steps to implement this constraint are as follows:
[0169] 1. Sort the candidate keyframe positions in chronological order.
[0170] 2. Starting from the first candidate keyframe position, check the time interval between it and the next candidate keyframe position.
[0171] 3. If the interval is less than 0.3 seconds, delete the candidate key frame position with the lower score.
[0172] 4. Repeat steps 2-3 until all candidate keyframe positions have been processed.
[0173] Finally, we try to match the detected candidate keyframe positions with predefined key actions (preparation, take-off, peak, landing). This step makes use of the key action-specific atoms added during the dictionary construction phase. We check if the most active atom at each candidate keyframe position corresponds to these predefined key actions. The matching process steps are as follows:
[0174] 1. For each candidate key frame position, find the element with the largest absolute value in its sparse representation coefficient.
[0175] 2. Check whether the dictionary atom corresponding to the element belongs to the predefined set of key action atoms.
[0176] 3. If it does, mark the candidate key frame position as the corresponding key action frame.
[0177] 4. If a predefined key action has no matching candidate key frame position, the candidate key frame position with the highest score within the corresponding time range can be selected as the key action frame.
[0178] Through this comprehensive analysis process, a set of candidate key frame positions {t_i|i=1, 2, ..., K} is finally obtained, where K is usually less than 10. Here, t_i represents the i-th candidate key frame position.
[0179] Step S160 , for the candidate key frame positions, evaluating the reconstruction error and sparsity corresponding to the candidate key frame positions.
[0180] For each candidate key frame position t_i, its reconstruction error and sparsity are calculated to evaluate the quality of the candidate key frame position as a key frame.
[0181] Calculation of reconstruction error E_recon(i): First, the overcomplete dictionary D constructed in step S130 and the sparse coefficient vector α_i are used to reconstruct the signal y_i=Dα_i. D is an m×n matrix, where m is the dimension of the feature vector and n is the number of atoms in the dictionary; α_i is an n×1 column vector, representing the sparse representation coefficient of the current frame. This reconstruction process can be understood as a linear combination of dictionary atoms, and the contribution of each atom is determined by the corresponding sparse representation coefficient. For example, if the jth element of α_i is 0.5, then the jth column of the dictionary D (i.e., the jth atom) will contribute to the reconstructed signal with a weight of 0.5. Next, the difference vector δ_i=x_i-y_i between the original feature vector x_i and the reconstructed signal y_i is calculated. Each element of δ_i reflects the reconstruction error on the corresponding dimension. A positive value indicates that the reconstructed signal is lower than the original signal on that dimension, and a negative value indicates that it is higher than the original signal. Finally, the reconstruction error is obtained by calculating the square of the L2 norm of the difference vector: E_recon(i)=δ_i^T·δ_i=Σ(δ_i,j)^2, where δ_i,j is the jth element of δ_i. This value provides a comprehensive measure of reconstruction quality, and the smaller the value, the more accurate the reconstruction.
[0182] The calculation of sparsity S(i) involves the statistics of non-zero elements in the sparse coefficient vector α_i. Considering the precision limitation of floating-point calculations, it is necessary to set a very small threshold ε (usually 10^-6) to determine whether an element is zero. Traverse each element α_i,j in α_i, and when |α_i,j|>ε, the counter is incremented by 1. This process can be implemented in a simple loop, or vectorized operations can be used to improve efficiency. The final value of the counter is S(i), which reflects the number of non-zero dictionary coefficients used to represent the frame. This process is actually calculating the approximate value of the "0 norm" of the vector. The smaller S(i), the sparser it is, which means that the frame can be effectively represented with fewer dictionary atoms.
[0183] In one example, the step of evaluating the reconstruction error and sparsity corresponding to the candidate key frame position, for the candidate key frame position, also includes the steps of: evaluating the preliminary reconstruction error and preliminary sparsity of the candidate key frame position; normalizing the preliminary reconstruction error to obtain a normalized reconstruction error value; normalizing the preliminary sparsity to obtain a normalized sparsity value; processing the normalized reconstruction error value and the normalized sparsity value based on the weight coefficient of the frame type to obtain the reconstruction error and sparsity.
[0184] Specifically, in order to make the evaluation results more comparable, the preliminary reconstruction error and preliminary sparsity are normalized. First, the maximum reconstruction error E_max and the maximum sparsity S_max in all candidate key frame positions are calculated, and then each frame is normalized: E_norm(i)=E_recon(i) / E_max, S_norm(i)=S(i) / S_max. This process maps both indicators to the [0,1] interval so that they can be compared under the same standard. After normalization, the smaller E_norm(i) is, the higher the reconstruction quality, and the smaller S_norm(i) is, the sparser it is.
[0185] Considering the importance difference of different types of key frames in standing long jump analysis, a weight coefficient w_type(i) based on frame type is introduced. For example, w_type=0.8 is set for take-off frame, w_type=1.0 for highest point frame, w_type=0.9 for landing frame, and w_type=1.2 for other frames. This setting reflects the different requirements for different types of frames: take-off and landing frames are given lower weights because they are crucial for long jump performance estimation, allowing larger reconstruction errors or lower sparsity; while the requirements for other frames are more stringent. Use these weights to adjust the normalized indicators: E_adjusted(i)=E_norm(i)*w_type(i), S_adjusted(i)=S_norm(i)*w_type(i). This adjustment allows different types of frames to be compared fairly based on their importance.
[0186] Through this series of calculations, an evaluation pair (E_adjusted(i), S_adjusted(i)) is finally obtained for each candidate keyframe position t_i. This evaluation pair comprehensively reflects the performance of the frame in terms of reconstruction accuracy and representation efficiency, while taking into account the importance of the frame type. A lower E_adjusted(i) value indicates that the frame can be more accurately represented by the dictionary, while a lower S_adjusted(i) value indicates that the frame can be effectively represented with fewer dictionary atoms.
[0187] Step S170 , when the relationship between the reconstruction error and the sparsity satisfies a preset condition, an optimal key frame position is determined from the candidate key frame positions based on the sparse representation coefficient, the adjusted reconstruction error and the sparsity.
[0188] Specifically, the preset conditions for the relationship between the reconstruction error and the sparsity are first defined. This condition reflects the requirements for the quality of the key frame, and the following conditions are preferably adopted:
[0189] C(i)=a*E_adjusted(i)+b*S_adjusted(i)≤threshold
[0190] Wherein, E_adjusted(i) is the adjusted reconstruction error of candidate key frame i, S_adjusted(i) is the adjusted sparsity of candidate key frame i, a and b are weight coefficients used to balance the importance of reconstruction error and sparsity, and threshold is a preset threshold.
[0191] For each candidate key frame i, calculate its C(i) value. If C(i)≤threshold, the candidate key frame meets the preset conditions and enters the next stage of consideration; otherwise, the candidate key frame is eliminated.
[0192] Next, for the candidate keyframes that meet the preset conditions, consider their sparse representation coefficients. Focus on the following features of the sparse coefficient vector α_i:
[0193] 1. Maximum coefficient value: max(|α_i|)
[0194] This value reflects the most significant feature intensity and is calculated by taking the element with the largest absolute value in α_i.
[0195] 2. Coefficient distribution: Use the Gini coefficient Gini(α_i) to measure the unevenness of the distribution
[0196] The calculation formula of the Gini coefficient is:
[0197] Gini(α_i)=(2*∑(j*|α_i,j|) / (n*∑|α_i,j|))-(n+1) / n
[0198] Where n is the dimension of α_i, α_i,j is the absolute value of the jth element in α_i, and the value of j ranges from 1 to n.
[0199] 3. Activation of specific atoms: atomic coefficient values related to key actions (such as take-off, highest point, landing)
[0200] A key atom set key_atoms is predefined, which contains the atom indexes related to key actions. The sum of the coefficient values corresponding to these atoms is calculated.
[0201] Combine these features into a sparse representation score:
[0202] SP(i)=w1*max(|α_i|)+w2*Gini(α_i)+w3*∑(α_i[k]forkinkey_atoms)
[0203] Among them, w1, w2, and w3 are weight coefficients.
[0204] The three main evaluation indicators are combined to form a comprehensive score:
[0205] Score(i) = λ1 * (1 - E_adjusted(i)) + λ2 * (1 - S_adjusted(i)) + λ3 * SP(i)
[0206] Among them, λ1, λ2, and λ3 are weight coefficients.
[0207] To ensure that the selected key frames are evenly distributed throughout the long jump process, a time distribution penalty term is introduced:
[0208] TP(i) = min(|t_i - t_j| for j in selected_frames) / T
[0209] Among them, t_i is the timestamp of the currently considered frame, selected_frames is the set of selected key frames, and T is the duration of the entire long jump process. This penalty term encourages the selection of frames that are far from the selected frames in terms of time.
[0210] The final scoring function is:
[0211] FinalScore(i) = Score(i) + λ4 * TP(i)
[0212] Among them, λ4 is the weight coefficient of the time distribution penalty term, and its value range is [0, 1]. λ4 can be set to 0.2.
[0213] Based on the above scoring function, the greedy algorithm is used to select the optimal key frames:
[0214] 1. Initialize an empty set S to store the selected key frames, and a list C to store all candidate key frames.
[0215] 2. Set the expected number of key frames to be selected N, for example, N = 5.
[0216] 3. When |S| < N and C is not empty, perform the following steps:
[0217] a. For each candidate frame i with C(i) ≤ threshold, calculate FinalScore(i).
[0218] b. Select the frame j with the highest FinalScore, remove it from C, and add it to S.
[0219] c. Update the time distribution penalty TP of the remaining frames:
[0220] For each remaining frame k in C, update its TP(k) = min(TP(k), |t_k - t_j| / T).
[0221] d. If |S|=N or there is no frame in C that satisfies C(i)≤threshold, exit the loop.
[0222] 4. Sort the frames in S in ascending order by timestamp t_i.
[0223] 5. Check whether key action frames (such as take-off frames and landing frames) are included in S:
[0224] a. If the take-off frame is missing, find the frame closest to the estimated take-off time in C and add it to S.
[0225] b. If the landing frame is missing, find the frame closest to the estimated landing time in C and add it to S.
[0226] c. If the number of frames in S exceeds N, the non-critical action frames with the lowest FinalScore need to be removed.
[0227] Through this process, the final optimal key frame position set is expressed as: S_opt={t_j|j=1,2,...,M}, where M is the number of key frames finally selected and t_j is the time position of the jth optimal key frame.
[0228] Step S1180: Based on the optimal key frame position, the key action frames of the long jump process are restored from the sparse representation coefficients using the compressed sensing technology. The key action frames include a take-off frame and a landing frame.
[0229] The core idea of compressed sensing technology is to recover the original signal from far fewer samples than required by the Nyquist sampling theorem, provided that the signal has a sparse representation under a certain basis or dictionary. In the analysis of standing long jump, this technology is reflected in the recovery of high-dimensional signals from sparse coefficients, low sampling rate reconstruction, and adaptive threshold processing. The application of compressed sensing makes it possible to reconstruct the complete signal from limited observations, which is particularly important when dealing with fast movements such as standing long jump, because it can effectively capture key information while reducing the processing of redundant data.
[0230] In one example, based on the optimal key frame position, the key action frames of the long jump process are sparsely recovered from the representation coefficients using compressed sensing technology, and the key action frames include the step of taking-off frames and landing frames, including the steps of: reconstructing the feature vectors of the optimal key frame position at each moment through matrix multiplication based on the sparse representation coefficients and the overcomplete dictionary; converting the reconstructed feature vectors into a reconstructed gradient field, and converting the reconstructed gradient into a reconstructed image sequence by solving the Poisson equation; in the reconstructed image sequence, selecting a first image subsequence of 2 frames before and after the estimated take-off moment, and a second image subsequence of 2 frames before and after the estimated landing moment; using sub-frame interpolation technology to improve The invention relates to a method for processing the first image subsequence and the second image subsequence after the time resolution is improved by using a motion change detection function to process the first image subsequence and the second image subsequence after the time resolution is improved, and respectively obtain a first motion change intensity sequence corresponding to the first image subsequence and a second motion change intensity sequence corresponding to the second image subsequence; the first motion change intensity sequence is processed by an adaptive threshold method to obtain a preliminary take-off frame, and the second motion change intensity sequence is processed by an adaptive threshold method to obtain a preliminary landing frame; the preliminary take-off frame is processed by a quadratic interpolation method, and the take-off frame is extracted from the processing result, and the preliminary landing frame is processed by a quadratic interpolation method, and the landing frame is extracted from the processing result.
[0231] Specifically, for each time t_j in the optimal key frame position set S_opt, the corresponding sparse coefficient vector α_j and the over-complete dictionary D constructed in step S130 are used to reconstruct the feature vector by matrix multiplication: x_j=Dα_j. This process can be understood as recombining the sparse representation coefficients into a complete feature description. The reconstructed feature vector x_j contains the key visual feature information of the frame image, but it is still in the feature space and has not yet been visualized.
[0232] Converting the reconstructed feature vector back to the image space is the next key step. Since the HOG (Histogram of Oriented Gradients) feature is used in step S130, the inverse transformation of the HOG feature is required. This process first rearranges the feature vector x_j into a 32x32x9 three-dimensional array, corresponding to the spatial and directional dimensions of the HOG feature. Subsequently, for each 8x8 cell, the local gradient field is reconstructed based on the gradient strength in 9 directions, which is actually reconstructing the edge and texture information in each small area.
[0233] The conversion from the reconstructed gradient field to the reconstructed image sequence is achieved by solving the Poisson equation. The Poisson equation is often used in image processing to reconstruct images from gradient fields. It can smoothly fill the interior of the region while maintaining edge information. Solving the Poisson equation is an iterative process. By setting appropriate boundary conditions and the number of iterations, a pixel value distribution close to the original image is gradually reconstructed. Finally, the reconstructed image is post-processed, including contrast adjustment and denoising, to improve image quality. The result of this series of steps is a reconstructed image sequence {Î_t}, where each Î_t is a reconstructed image at the corresponding time t.
[0234] In order to accurately locate the take-off and landing frames, two frames before and after the estimated take-off and landing moments are selected to form two sequences of 5 frames (i.e., the first image subsequence and the second image subsequence). The above reconstruction process is performed on each frame in these two subsequences to obtain a high-quality local image sequence. This local refinement strategy ensures that the real key moment will not be missed due to estimation errors.
[0235] To further improve the temporal resolution, a sub-frame interpolation technique is introduced. The sparse coefficients of the intermediate frames are generated by linear interpolation: α_interp(t)=(1-λ)α_j+λα_{j+1}, where λ∈[0,1] is the interpolation coefficient, and j and j+1 are adjacent key frame indices. This interpolation method assumes that the motion between adjacent frames is linear, which is a reasonable approximation for small time intervals. Interpolated frames are generated with a step size of 0.1 frames, which means that 9 interpolated frames are created between every two original frames. The reconstruction steps described previously are repeated for each interpolated frame, that is, the feature vector is first reconstructed by matrix multiplication, and then the inverse transform of the HOG feature is performed to convert the feature vector back to the image space.
[0236] With a high-temporal-resolution image sequence, a method is needed to accurately locate the moments of takeoff and landing. To this end, an image difference-based motion change detection function D(t) is defined. This function calculates the sum of pixel differences between adjacent frames: D(t)=∑_{x,y∈R}|Î_t(x,y)-Î_{t-1}(x,y)|, where R is the region of interest. For the takeoff frame, R focuses on the toe area; for the landing frame, R includes the entire body contour. Applying this function to all interpolated frames, a motion change intensity sequence is obtained, which reflects the degree of motion change at each moment during the movement.
[0237] An adaptive threshold method is used to identify the real critical moment from the sequence of action change intensity. First, the mean μ and standard deviation σ of all D(t) values are calculated, and then the threshold θ=μ+2σ is set. The choice of the mean plus twice the standard deviation as the threshold is based on the characteristics of the normal distribution. This threshold can usually capture changes that are significantly higher than the average level. For the take-off moment, scan backward from the beginning of the sequence to find the first moment t_takeoff (preliminary take-off frame) that satisfies D(t)>θ; and for the landing moment, scan forward from the end of the sequence to find the last moment t_landing (preliminary landing frame) that satisfies D(t)>θ. This two-way scanning method can effectively locate the beginning and end of the action.
[0238] To further improve the accuracy of positioning, local quadratic interpolation is performed before and after the moment that meets the conditions. Specifically, a quadratic function is fitted in a small range near t_takeoff and t_landing, and the local maximum point of this function is found. This maximum point is considered to be the moment when the action changes most dramatically, so it is selected as the final precise moment. This refined processing can find more precise moments between discrete time points, thereby achieving sub-frame-level temporal accuracy.
[0239] Finally, the take-off frame Î_takeoff and the landing frame Î_landing are extracted at precisely located moments t_takeoff and t_landing. These two frames are the key outputs of the entire analysis process, not only containing the key information needed to estimate the long jump distance, but also providing important visual data for subsequent kinematic analysis. The toe position in the take-off frame defines the starting point of the long jump, while the position of the first contact of the body in the landing frame defines the end point of the long jump. With this approach, a temporal accuracy of 0.1 frames is achieved, which is much higher than the frame rate of conventional videos and can capture subtle changes in movements that are difficult to distinguish with the naked eye. At the same time, the spatial accuracy of the reconstructed image is comparable to the resolution of the original image, ensuring the accuracy of subsequent analysis.
[0240] Step S190, locating the key points of the take-off frame and the key points of the landing frame, and calculating the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired field calibration information;
[0241] In one example, the steps of locating the key points of the take-off frame and the landing frame, and calculating the spatial coordinates of the take-off point and the landing point of a standing long jumper in combination with the pre-acquired site calibration information include the following steps: using the site calibration information to establish a mapping relationship between the image coordinate system and the spatial coordinate system; calculating the take-off image coordinates of the key points of the take-off frame, and calculating the landing image coordinates of the key points of the landing frame; based on the mapping relationship, converting the take-off image coordinates into the spatial coordinates of the take-off point, and converting the landing image coordinates into the spatial coordinates of the landing point.
[0242] Specifically, the mapping relationship between the image coordinate system and the space coordinate system is first established using the pre-acquired site calibration information, which usually includes the camera's intrinsic parameter matrix K and extrinsic parameter matrix [R|t], as well as the actual size of the site and the coordinates of the reference point.
[0243] The camera's intrinsic parameter matrix K contains information such as focal length, principal point coordinates, and pixel size, while the extrinsic parameter matrix [R|t] describes the position and orientation of the camera in the spatial coordinate system. These parameters are obtained through the camera calibration process, which usually uses standard objects such as a chessboard, takes images at multiple locations in the long jump field, and then uses the calibration algorithm to calculate them.
[0244] Based on the above calibration information, the mapping relationship between the image coordinate system and the space coordinate system is established, which can be expressed as:
[0245] s[u,v,1]^T=K[R|t][X,Y,Z,1]^T
[0246] Where [u,v] are image coordinates, [X,Y,Z] are spatial coordinates, and s is the scale factor.
[0247] Next, you need to accurately locate key points in Î_takeoff and Î_landing. For the take-off frame Î_takeoff, you need to locate the position of the athlete's toes; for the landing frame Î_landing, you need to locate the position where the athlete's body first touches the sand pit.
[0248] When processing the take-off frame, a series of image processing techniques are used to locate the toe position. First, an edge detection algorithm is used to identify all the edges in the image, just like outlining the outline of all objects in the image with a pencil. Then, a complete outline is found among these edges. Next, attention is focused on the pre-defined take-off area, which is usually a rectangular area at the bottom of the image. Within this area, the contour point closest to the take-off line is found, which is likely to be the athlete's toe position. In this way, the toe position is successfully found in the image and represented by the image coordinates (u_takeoff, v_takeoff).
[0249] For the landing frame, the situation is a little more complicated, because the athlete may first touch the bunker with a different part of the body. Background subtraction techniques are used to separate the athlete's outline, as if "cutting" the athlete out of the background. In order to make the result more accurate, some additional processing is also performed, such as removing possible noise and filling possible small holes in the outline. Then, the lowest point of the athlete's outline is found in the predefined landing area. This point is likely to be the location where the athlete's body first touches the bunker. The image coordinates (u_landing, v_landing) are used to represent this landing point.
[0250] Through the above steps, the image coordinates of the take-off point and the landing point in the image are obtained, which are represented as (u_takeoff, v_takeoff) and (u_landing, v_landing) respectively.
[0251] Once the image coordinates of these key points are determined in the image, they can be converted into spatial coordinates using the previously established coordinate mapping relationship. However, since there is only 2D image information, directly solving the 3D spatial coordinates is an underdetermined problem. To solve this problem, the characteristics of the long jump field can be used to assume that the take-off point and the landing point are on the same plane (usually the ground or the surface of the bunker).
[0252] Specifically, let the equation of the ground plane be aX+bY+cZ+d=0, and substitute it into the previous coordinate mapping equation to obtain:
[0253] s[u,v,1]^T=K[R|t][X,Y,-((aX+bY+d) / c),1]^T
[0254] Where K is the camera's intrinsic matrix, and [R|t] is the camera's extrinsic matrix. R is a 3x3 rotation matrix that describes the camera's orientation, and t is a 3x1 translation vector that describes the camera's position in the world coordinate system. Expanding R can be written as:
[0255] s[u,v,1]^T=K[r1,r2,r3,t][X,Y,-((aX+bY+d) / c),1]^T
[0256] Where r1, r2, r3 are the three column vectors of the rotation matrix R. Through this equation, the homography matrix H between the image coordinates and the ground coordinates can be established:
[0257] H=K[r1,r2,t-r3((aX+bY+d) / c)]
[0258] Using this homography matrix, we can transform a point [u,v] in the image to a point [X,Y] on the ground:
[0259] [X,Y,1]^T=H^(-1)[u,v,1]^T
[0260] Finally, the spatial coordinates of the take-off point and the landing point are:
[0261] Take-off point: (X1, Y1, Z1) = H^(-1)[u_takeoff, v_takeoff, 1]^T
[0262] Landing point: (X2, Y2, Z2) = H^(-1)[u_landing, v_landing, 1]^T
[0263] Through the above steps, the key points in the 2D image are successfully converted into 3D space coordinates.
[0264] Step S111, based on the spatial coordinates of the standing long jumper's take-off point and the spatial coordinates of the landing point, the long jump distance of the standing long jumper is calculated.
[0265] The three-dimensional coordinates (X1, Y1, Z1) of the take-off point and the three-dimensional coordinates (X2, Y2, Z2) of the landing point calculated in step S190 are obtained. These coordinates are expressed in a pre-calibrated spatial coordinate system in meters. In this coordinate system, the X-axis represents the main direction of the long jump, the Y-axis is perpendicular to the long jump direction and represents the lateral offset, and the Z-axis represents the height.
[0266] The long jump distance is calculated using the following formula: distance = X2-X1. This formula gives the distance projected on the X axis from the take-off point to the landing point. The reason for choosing the X-axis projection distance instead of the straight-line distance is that in the standard standing long jump measurement, only the horizontal distance is considered, ignoring the height difference and lateral offset. This calculation method is consistent with the traditional manual measurement method, ensuring the comparability of the results.
[0267] To ensure measurement accuracy, a series of verification steps are performed:
[0268] 1. Verify the legitimacy of the take-off point location:
[0269] Determine whether the X coordinate of the take-off point meets the condition: |X1|≤δ1, where δ1 is a preset threshold, preferably 0.1 meters. The purpose of this verification step is to ensure that the student's take-off position is within the allowed range to prevent "jumping" or incorrect take-off line determination. If this condition is not met, the potential problem is marked in the measurement result to remind you that further inspection or re-measurement is required.
[0270] 2. Ensure that the landing point is within the effective measurement range:
[0271] Determine whether the following conditions are met at the same time: X2>X1 and X2≤L, where L is the effective length of the long jump field, usually 4 meters. The purpose of this verification step is to ensure the rationality of the measurement results and prevent abnormal results caused by coordinate calculation errors or other reasons. If the landing point coordinates are smaller than the take-off point coordinates, or exceed the effective range of the long jump field, the coordinate calculation process will be rechecked.
[0272] 3. Calculate lateral offset:
[0273] Y_offset = |Y2-Y1|. If Y_offset exceeds a preset threshold δ2 (e.g. 0.5 m), it is flagged in the measurement result for further inspection. The calculation and verification of the lateral offset is very helpful for evaluating the students’ jumping trajectory and technique, and can also be used to detect possible measurement errors.
[0274] 4. Calculate the height difference:
[0275] Z_offset=|Z2-Z1|. If Z_offset exceeds the preset threshold δ3 (for example, 0.1 meters), it may indicate that there is a problem with the height measurement or the long jump field is not flat enough, which needs to be marked in the measurement results. The calculation of the height difference is mainly used to verify the accuracy of the measurement and the flatness of the field. The height factor is usually not considered in the standard standing long jump measurement.
[0276] After the above verification steps are completed and all checks are passed, a measurement record with detailed information is generated. The measurement record is stored in a database for subsequent analysis and query. At the same time, the measurement results, including the long jump distance and any relevant warnings, are displayed in real time through the display device. This real-time feedback not only allows students and teachers to understand the results immediately, but also to detect potential measurement problems in time, improving the efficiency and reliability of the entire testing process.
[0277] This method ensures high accuracy and reliability of the measurement results through precise three-dimensional coordinate calculation and multiple verification steps. The entire measurement process can be completed automatically within seconds, significantly improving measurement efficiency, and is particularly suitable for scenarios that require a large number of measurements, such as physical education courses or large-scale physical fitness tests.
[0278] The present invention provides a method for extracting key frames and estimating performance of standing long jump based on sparse representation, and comprises the following steps: obtaining first calibrated long jump video data of a standing long jumper, and determining the motion feature distribution corresponding to the calibrated long jump video data; pre-classifying the calibrated long jump video data using the motion feature distribution to obtain a pre-classification result, and dividing the test site into multiple functional areas according to the pre-classification result; obtaining second calibrated long jump video data based on the functional areas, and constructing an overcomplete dictionary of the standing long jumper according to the second calibrated long jump video data; collecting measured standing long jump video data, and sparsely encoding the measured standing long jump video data based on the overcomplete dictionary to obtain a sparse representation coefficient; selecting multiple candidate key frames in the measured standing long jump video data according to the sparse representation coefficient. Frame position; for the candidate key frame position, evaluate the reconstruction error and sparsity corresponding to the candidate key frame position; when the relationship between the reconstruction error and the sparsity meets the preset conditions, based on the sparse representation coefficient, the adjusted reconstruction error and sparsity, determine the optimal key frame position from the candidate key frame position; based on the optimal key frame position, use compressed sensing technology to recover the key action frame of the long jump process from the sparse representation coefficient, and the key action frame includes the take-off frame and the landing frame; locate the key points of the take-off frame and the landing frame, and calculate the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired site calibration information; based on the spatial coordinates of the take-off point and the landing point of the standing long jumper, measure the long jump distance of the standing long jumper. This application ensures the high accuracy and reliability of the measurement results through accurate three-dimensional coordinate calculation and multiple verification steps. The entire measurement process can be completed automatically within a few seconds, which significantly improves the measurement efficiency, and is particularly suitable for scenarios where a large number of measurements are required, such as physical education courses or large-scale physical fitness tests.
[0279] 1. High-precision motion capture and analysis: This application achieves accurate reconstruction of key frames of the long jump process by constructing an overcomplete dictionary of standing long jump actions and applying sparse coding technology. In particular, using the principle of compressed sensing and sub-frame interpolation technology, the system can accurately locate the take-off and landing moments with a time accuracy of 0.1 frames. This high-temporal-resolution motion capture capability, combined with an image-differentiated motion change detection function, enables the system to accurately identify and analyze rapidly changing motion details, greatly improving the accuracy of standing long jump measurements.
[0280] 2. Intelligent test site division and adaptive data collection: This application innovatively introduces a method for dividing the functional areas of the test site based on the pre-classification of the initial long jump data. Through cluster analysis of the long jump mode, the system can automatically adjust the camera layout, measurement area division and data collection point settings to adapt to different types of long jump action characteristics. This intelligent site configuration not only improves the pertinence and efficiency of data collection, but also optimizes the measurement parameters according to different long jump modes (such as "short distance high arc type", "long distance low arc type", etc.), ensuring the best measurement results in various long jump modes.
[0281] 3. Robust multi-dimensional long jump parameter estimation: This application establishes a robust long jump parameter estimation system by comprehensively using HOG feature extraction, multi-scale dictionary learning and adaptive threshold processing. The system can not only accurately measure the long jump distance, but also reliably estimate key technical indicators such as take-off angle and take-off speed. In particular, through the dynamic evaluation of reconstruction error and sparsity, combined with the time distribution penalty term, the system can effectively handle complex situations such as motion blur and partial occlusion while ensuring measurement accuracy.
[0282] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0283] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 2 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store long jump video data. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a method for extracting key frames and estimating performance of standing long jump based on sparse representation.
[0284] Those skilled in the art will understand that Figure 2 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0285] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0286] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0287] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.
Claims
1. A method for extracting key frames and estimating performance of standing long jump based on sparse representation, characterized in that: The following steps are involved: Acquire first calibrated long jump video data of a standing long jumper, and determine a motion feature distribution corresponding to the first calibrated long jump video data; Pre-classifying the calibration long jump video data using the motion feature distribution to obtain a pre-classification result, and dividing the test site into a plurality of functional areas according to the pre-classification result; the pre-classification result at least includes a short-distance high arc type, a long-distance low arc type, and a medium-distance balanced type; Based on the functional area, obtaining second calibrated long jump video data, and constructing an overcomplete dictionary of standing long jumpers according to the second calibrated long jump video data; the overcomplete dictionary contains typical action frames corresponding to the functional area; Collecting measurement standing long jump video data, and sparsely encoding the measurement standing long jump video data based on the overcomplete dictionary to obtain sparse representation coefficients; Selecting a plurality of candidate key frame positions in the measured standing long jump video data according to the sparse representation coefficients; For the candidate key frame position, evaluating the reconstruction error and sparsity corresponding to the candidate key frame position; In a case where the relationship between the reconstruction error and the sparsity satisfies a preset condition, determining an optimal key frame position from the candidate key frame positions based on the sparse representation coefficient, the adjusted reconstruction error and the sparsity; Based on the optimal key frame position, using compressed sensing technology to recover key action frames of the long jump process from the sparse representation coefficients, the key action frames including a take-off frame and a landing frame; Locating the key points of the take-off frame and the key points of the landing frame, and calculating the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired site calibration information; Calculating the long jump distance of the standing long jumper based on the spatial coordinates of the take-off point and the spatial coordinates of the landing point of the standing long jumper; Among them, based on the optimal key frame position, using compressed sensing technology to sparsely recover the key action frames of the long jump process from the representation coefficients, wherein the key action frames include a take-off frame and a landing frame, the steps include: Reconstructing the feature vector of each moment of the optimal key frame position through matrix multiplication based on the sparse representation coefficient and the overcomplete dictionary; Converting the reconstructed feature vector into a reconstructed gradient field, and converting the reconstructed gradient into a reconstructed image sequence by solving the Poisson equation; In the reconstructed image sequence, a first image subsequence of 2 frames before and after the estimated take-off moment and a second image subsequence of 2 frames before and after the estimated landing moment are selected; Using sub-frame interpolation technology to improve the temporal resolution of the first image subsequence and the second image subsequence; Processing the first image subsequence and the second image subsequence with improved time resolution using a motion change detection function to obtain a first motion change intensity sequence corresponding to the first image subsequence and a second motion change intensity sequence corresponding to the second image subsequence, respectively; The first action change intensity sequence is processed by an adaptive threshold method to obtain a preliminary take-off frame, and the second action change intensity sequence is processed by an adaptive threshold method to obtain a preliminary landing frame; The preliminary take-off frame is processed by a quadratic interpolation method, and the take-off frame is extracted from the processing result; and the preliminary landing frame is processed by a quadratic interpolation method, and the landing frame is extracted from the processing result.
2. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of obtaining the calibrated long jump video data of the standing long jumper and determining the motion feature distribution corresponding to the calibrated long jump video data comprises the steps of: Preprocessing the calibration long jump video data to obtain single calibration long jump video sub-data; the calibration long jump video sub-data includes a complete long jump action of a standing long jumper; Extract key motion features from the calibration long jump video sub-data; the motion features include take-off angle, take-off speed, aerial posture, landing skills and jump distance; Processing the motion features using image processing technology to quantify the motion features; Perform statistical analysis on the quantified motion features to obtain the motion feature distribution.
3. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of pre-classifying the calibration long jump video data using the motion feature distribution to obtain a pre-classification result, and dividing the test site into a plurality of functional areas according to the pre-classification result comprises the steps of: Calculating the mean value and standard deviation of the motion characteristics of the calibration long jump video data, and obtaining a coefficient of variation according to the mean value and the standard deviation; Performing feature normalization on the maximum and minimum values of the motion features of the calibrated long jump video data to obtain a normalized result; Based on the coefficient of variation, the normalization result and the motion feature distribution, obtaining aggregation energy through experiments and cross-validation; The pre-classification result is obtained according to the aggregated energy, and the test site is divided into a plurality of functional areas according to the pre-classification result.
4. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of acquiring second calibrated long jump video data based on the functional area and constructing an overcomplete dictionary of standing long jumpers according to the second calibrated long jump video data comprises the steps of: According to the functional area, optimizing the data acquisition strategy to obtain the second calibrated long jump video data; Preprocessing the second calibrated long jump video data to adjust pixels of all frames of the second calibrated long jump video data to MxN pixels; the preprocessing includes image denoising, contrast adjustment and size standardization; extracting a feature vector from the preprocessed second calibrated long jump video data using a histogram of oriented gradients feature descriptor, and performing dimensionality reduction processing on the feature vector using a principal component analysis algorithm; Using K-means clustering algorithm to process the feature vector after dimension reduction to construct an initial dictionary; The initial dictionary is optimized and refined using a K singular value decomposition algorithm to obtain the overcomplete dictionary.
5. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of collecting the measured standing long jump video data and sparsely encoding the measured standing long jump video data based on an overcomplete dictionary to obtain a sparse representation coefficient includes the steps of: Preprocessing the standing long jump measurement video data; Using a histogram of oriented gradients feature descriptor, extracting a feature vector from the preprocessed measured standing long jump video data; Performing dimensionality reduction processing on the feature vector using a principal component analysis algorithm to obtain the feature vector after dimensionality reduction having the same dimension as the overcomplete dictionary; The orthogonal matching pursuit algorithm is used to optimize the feature vector after dimension reduction to obtain the sparse representation coefficient.
6. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of selecting a plurality of candidate key frame positions in the standing long jump measurement video data according to the sparse representation coefficient comprises the steps of: Calculating the sparsity of each frame in the standing long jump measurement video data, and calculating the sparsity first-order difference of the sparsity; Calculating the energy of each frame in the standing long jump measurement video data, and calculating the first-order difference of the energy; Calculating the atomic activity of the standing long jump video data in time sequence; Calculating the cosine similarity of the sparse representation coefficients of adjacent frames in the standing long jump measurement video data; Constructing a comprehensive score function according to the first-order difference of sparsity, the first-order difference of energy, the atomic activity and the cosine similarity; Based on the comprehensive score function, a peak detection algorithm is used to determine the candidate key frame positions in the measured standing long jump video data.
7. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of evaluating the reconstruction error and sparsity corresponding to the candidate key frame position further includes the following steps: For the candidate key frame position, evaluating a preliminary reconstruction error and a preliminary sparsity of the candidate key frame position; Normalizing the preliminary reconstruction error to obtain a normalized reconstruction error value; Normalizing the preliminary sparsity to obtain a normalized sparsity value; The reconstruction error normalization value and the sparsity normalization value are processed based on the weight coefficient of the frame type to obtain the reconstruction error and the sparsity.
8. The method for extracting key frames and estimating performance of standing long jump based on sparse representation according to claim 1, characterized in that: The step of locating the key points of the take-off frame and the key points of the landing frame, and calculating the spatial coordinates of the take-off point and the landing point of the standing long jumper in combination with the pre-acquired site calibration information, comprises the steps of: Establishing a mapping relationship between an image coordinate system and a space coordinate system using the site calibration information; Calculate the take-off image coordinates of the key points of the take-off frame, and calculate the landing image coordinates of the key points of the landing frame; Based on the mapping relationship, the take-off image coordinates are converted into the spatial coordinates of the take-off point, and the landing image coordinates are converted into the spatial coordinates of the landing point.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.
Citation Information
Patent Citations
Method for extracting key frame of human motion sequence
CN108520250A
A video compressed sensing reconstruction method based on dynamic selection of multi-hypothesis matching blocks
CN109934882A
Video super-resolution reconstruction method fusing motion features
CN112580473A
Key frame detection method and device, equipment, storage medium and program product
CN114359775A
Long jump measuring method based on key points of human skeleton
CN118038549A
