Rock climbing action analysis system and method based on multi-dimensional quantitative evaluation

Abstract

The invention discloses a rock climbing action analysis system and method based on multi-dimensional quantitative evaluation, and relates to the technical field of computer vision, and the method comprises the steps: carrying out the spatial registration and skeleton node positioning of a multi-mode rock climbing data set through employing an ICP algorithm in combination with an OpenPose algorithm, forming a skeleton motion state matrix, and carrying out the calculation of the skeleton motion state matrix; inputting the skeleton motion state matrix into a skeleton motion multi-dimensional quantitative model, performing time sequence segmentation and dynamic behavior classification by a time sequence analysis layer, performing multi-dimensional index calculation and weight distribution by an action evaluation layer, outputting a rock climbing action comprehensive score, performing differential operation on the rock climbing action comprehensive score and a standard action score, and obtaining a quantitative offset tensor. According to the invention, through the ICP algorithm, the OpenPose algorithm and the skeleton motion multi-dimensional quantitative model, the spatial registration precision of the multi-modal rock climbing data is improved, and fine dynamic classification and multi-dimensional index evaluation of rock climbing actions are realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a climbing motion analysis system and method based on multi-dimensional quantitative evaluation. Background Technology

[0002] With the continuous advancement of sports science and computer technology, rock climbing, as an extreme sport, has gradually gained widespread attention, and its training and assessment methods increasingly rely on advanced technological means. In recent years, with the help of computer vision, pattern recognition, and data processing technologies, researchers have developed a variety of methods for analyzing rock climbing, which can capture athletes' movement trajectories and conduct multi-dimensional quantitative assessments, providing a scientific basis for training.

[0003] Traditional climbing motion analysis relies heavily on coaches' experience and simple video playback, which has several shortcomings. First, traditional methods struggle with efficient and accurate spatial registration and skeletal node localization when processing multimodal climbing data, resulting in low-quality foundational data for subsequent analysis. Second, existing motion assessment models typically focus only on static indicators or simple time-series segmentation, lacking in-depth exploration of dynamic behavior classification and multidimensional indicator calculation. This fails to fully reflect the complexity and diversity of climbing movements, directly impacting the accuracy of climbing motion analysis. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a climbing motion analysis method based on multi-dimensional quantitative evaluation to solve the problems of low spatial registration accuracy of multimodal climbing data and insufficient accuracy of climbing motion analysis.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: In a first aspect, the present invention provides a climbing motion analysis method based on multi-dimensional quantitative evaluation, which includes using the ICP algorithm combined with the OpenPose algorithm to perform spatial registration and skeletal node localization on a multimodal climbing dataset to form a skeletal motion state matrix. The skeletal motion state matrix is ​​input into the skeletal motion multidimensional quantification model. The time series analysis layer performs time series segmentation and dynamic behavior classification, and the action evaluation layer performs multidimensional index calculation and weight allocation to output a comprehensive score of rock climbing action. The comprehensive score and standard score of rock climbing action are differentially calculated to obtain the quantized offset tensor. The quantized offset tensor and skeletal motion parameters are then subjected to feature mapping and transformation superposition through improved B-spline surface fitting to generate a skeletal superposition comparison view. Wavelet transform is used to extract climbing phase features from the superimposed comparison view of bones, and climbing actions are analyzed in real time based on the climbing phase features.

[0007] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the multimodal climbing dataset includes raw climbing RGB-D data, skeletal motion parameters, and rock wall environmental parameters.

[0008] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the step of obtaining the skeletal motion feature matrix specifically includes the following steps. The ICP algorithm is applied to spatially register the multimodal climbing dataset to obtain a spatiotemporally synchronized dataset; temporal domain analysis is performed on the spatiotemporally synchronized dataset to form motion frequency domain features; The OpenPose algorithm was applied to perform skeletal node localization on a spatiotemporal synchronization dataset to obtain a sequence of skeletal node coordinates. The motion frequency domain features and the bone node coordinate sequence are fused across modes to construct a bone motion feature matrix.

[0009] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the output of the comprehensive climbing motion score specifically includes the following steps. A temporal analysis layer and a motion evaluation layer are built, and deformable convolution and residual connections are applied for parameterized stacking to construct a multidimensional quantitative model of skeletal motion. The skeletal motion feature matrix is ​​input into the skeletal motion multidimensional quantization model. The time series analysis layer uses bidirectional LSTM units to perform time series segmentation and dynamic behavior classification, and outputs the current action stage identifier. The action evaluation layer uses a spatiotemporal attention mechanism to perform multi-dimensional index calculation and weight allocation on the current action stage identifier to obtain a four-dimensional score value. The current action stage identifier and the four-dimensional score are concatenated in the feature channel to generate a comprehensive feature vector; The weighted summation method is used to perform weight allocation and aggregation on the comprehensive feature vector to form a comprehensive score for rock climbing actions.

[0010] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the step of obtaining the quantitative offset tensor specifically includes the following steps. The climbing action comprehensive score and standard action score are differentially calculated using Gaussian integral, and the range is normalized using Min-Max standardization to generate the original offset value; the original offset value is then piecewise linearly quantized to obtain the quantized offset tensor. The standard action scoring is based on a tiered zoning rule definition.

[0011] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the generation of the skeletal overlay comparison view specifically includes the following steps. Multi-scale stitching of the quantized offset tensor and skeletal motion parameters is performed to obtain the spatiotemporal feature tensor; An improved B-spline surface fitting method is used to perform feature mapping on the spatiotemporal feature tensor to generate the skeletal motion basis surface; The ARAP deformation method is applied to transform and superimpose the basic surface of the skeleton motion, and the norm smoothing operator is used for regularization constraints to form a skeleton superimposed comparison view.

[0012] As a preferred embodiment of the climbing motion analysis method based on multi-dimensional quantitative evaluation described in this invention, the step of performing real-time analysis of climbing motions based on climbing phase characteristics specifically includes the following steps. Wavelet transform is used to perform multi-level decomposition and phase angle quantization on the superimposed skeleton comparison view to extract climbing phase features; Based on climbing phase characteristics, the analytic hierarchy process (AHP) is used to perform real-time analysis of joint coordination and motion coordination quality in rock climbing movements, generate quantitative motion data, integrate the quantitative motion data, and output a real-time analysis report.

[0013] Secondly, the present invention provides a climbing motion analysis system based on multi-dimensional quantitative evaluation, including a registration and fusion module, which uses the ICP algorithm combined with the OpenPose algorithm to perform spatial registration and skeletal node localization on a multimodal climbing dataset to form a skeletal motion state matrix. The multidimensional quantization module inputs the skeletal motion state matrix into the skeletal motion multidimensional quantization model, the time series analysis layer performs time series segmentation and dynamic behavior classification, and the action evaluation layer performs multidimensional index calculation and weight allocation, outputting a comprehensive score for rock climbing actions. The dynamic mapping module performs differential calculations on the comprehensive score and standard action score of rock climbing action to obtain the quantized offset tensor. Through improved B-spline surface fitting, feature mapping and transformation superposition are performed on the quantized offset tensor and skeletal motion parameters to generate a skeletal superposition comparison view. The real-time analysis module uses wavelet transform to extract climbing phase features from the skeleton overlay comparison view, and performs real-time analysis of climbing actions based on the climbing phase features, outputting a real-time analysis report.

[0014] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the climbing motion analysis method based on multi-dimensional quantitative evaluation as described in the first aspect of the present invention.

[0015] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the climbing motion analysis method based on multi-dimensional quantitative evaluation as described in the first aspect of the present invention.

[0016] The beneficial effects of this invention are as follows: by combining the ICP algorithm and the OpenPose algorithm, the spatial registration accuracy of multimodal climbing data is significantly improved, ensuring the accuracy and stability of skeletal node positioning; at the same time, a multidimensional quantitative model of skeletal motion, including a temporal analysis layer and a motion evaluation layer, is constructed, realizing fine dynamic classification and multidimensional index evaluation of climbing motions, thereby effectively improving the overall accuracy and scientific nature of motion analysis. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a climbing motion analysis method based on multi-dimensional quantitative evaluation.

[0019] Figure 2 This is a schematic diagram of a climbing motion analysis system based on multi-dimensional quantitative evaluation.

[0020] Figure 3 A flowchart for constructing the skeletal motion state matrix.

[0021] Figure 4 A flowchart for generating a comprehensive score for rock climbing movements. Detailed Implementation

[0022] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0023] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0025] Reference Figures 1-4 This is one embodiment of the present invention, which provides a climbing motion analysis method based on multi-dimensional quantitative evaluation, including the following steps: S1. Use the ICP algorithm combined with the OpenPose algorithm to perform spatial registration and skeletal node localization on the multimodal climbing dataset to form a skeletal motion state matrix.

[0026] Specifically, the operations include the following: S1.1 Collect a multimodal climbing dataset, which includes raw RGB-D climbing data, skeletal motion parameters, and rock wall environmental parameters.

[0027] The raw RGB-D data for rock climbing includes depth images and color video, acquired through RGB-D cameras installed at the climbing area; Skeletal motion parameters include joint angles, limb length, and motion trajectory; joint angles are acquired using wearable inertial measurement unit (IMU) sensors, limb length is measured using optical marker units, and motion trajectory is acquired using high-precision cameras; The environmental parameters of the rock wall include the surface roughness, temperature, and humidity. The surface roughness of the rock wall is collected by a tactile sensor, the temperature is collected by an embedded thermometer, and the humidity is collected by a humidity sensor. Multimodal climbing datasets can support big data processing and analysis, laying the foundation for subsequent dynamic behavioral classification of climbing actions and calculation of multidimensional indicators.

[0028] S1.2. Preprocessing of the multimodal climbing dataset: Specifically, for the raw RGB-D climbing data, median filtering is used with a fixed-size window (e.g., 7×7) for smoothing to remove impulse noise from the depth image. Histogram equalization is used to enhance the contrast of the depth image, improving the visibility of joint contours in low-light conditions. Simultaneously, white balance is applied to correct the color video to ensure color accuracy. For skeletal motion parameters, mean smoothing is performed using a sliding window to eliminate dimensional differences. A Butterworth low-pass filter is used to perform high-frequency filtering on the skeletal motion parameters to suppress high-frequency electromyography interference. For rock wall environmental parameters, DBSCAN clustering is used to detect outliers, identify and remove them, and Gaussian filtering is applied simultaneously for spatial smoothing and noise reduction to achieve data stability. Dynamic time warping is applied to align the preprocessed RGB-D raw climbing data, skeletal motion parameters, and rock wall environmental parameters for temporal alignment. Principal component analysis (PCA) is then applied for feature dimensionality reduction to improve computational efficiency. Finally, Z-score normalization is used to output the preprocessed multimodal climbing dataset.

[0029] S1.3. Apply the ICP algorithm to spatially register the multimodal climbing dataset to obtain a spatiotemporally synchronized dataset. Perform temporal domain analysis on the spatiotemporally synchronized dataset to form motion frequency domain features. Specifically, use Fast Fourier Transform to perform spectral energy decomposition on the multimodal climbing dataset to extract motion spectral features. Use the sliding window statistical method to perform peak detection and energy integration on the motion spectral features to obtain macroscopic rhythmic features of the movements. Use the ICP algorithm to perform spatial alignment and temporal matching on the macroscopic rhythmic features of the movements to form registered motion features. Perform feature recombination and matrix filling on the registered motion features to obtain the motion feature matrix. Apply principal component analysis to compress the dimensions and weighted integrate the motion feature matrix to obtain the spatiotemporally synchronized dataset. The Butterworth filter was applied to bandpass filter the spatiotemporal synchronization dataset to remove high-frequency noise and low-frequency drift. Then, the sliding short-time Fourier transform was used for joint time-frequency decomposition to obtain the time-frequency distribution features. Energy normalization was applied to perform energy calibration and scale adjustment on the time-frequency distribution features to obtain a standardized time-frequency domain energy distribution. The standardized time-frequency domain energy distribution was then finely decomposed into frequency bands using Morlet wavelet transform to extract the action rhythm signal. Combined with wavelet packet decomposition, sub-bands were divided, dividing the action rhythm signal into 8 sub-bands. Simultaneously, energy proportion statistics were applied to quantize the feature intensity of the 8 sub-bands to form micro-force details. Feature fusion and weight allocation are performed on the macroscopic rhythmic features and microscopic force details of the motion, and linear discriminant analysis is used for feature selection and dimensionality reduction to obtain the fused motion representation vector. The fused representation vector is then normalized by Z-score standardization to generate motion frequency domain features.

[0030] S1.4. Apply the OpenPose algorithm to perform skeletal node localization on the spatiotemporal synchronization dataset to obtain the skeletal node coordinate sequence. Specifically, bicubic interpolation is used to upsample the spatiotemporal synchronization dataset to extract high-resolution image data, and histogram equalization (CLAHE) is applied to perform contrast enhancement to obtain initial visual features. The initial visual features are then forward-propagated frame by frame using gradient descent, and a ReLU activation function is applied for nonlinear mapping to generate high-dimensional image features. The OpenPose algorithm is used to perform skeletal node localization on the high-dimensional image features to obtain initial detection joints. Spatial probability mapping is then performed on the initial detection joints to form a sequence containing 1 A confidence heatmap of 8 joints is generated. Vector regression is applied to perform nonlinear fitting on the high-dimensional image features to obtain the Part Affinity Field (PAF). Subsequently, a bipartite graph matching algorithm is used to perform correlation matching and topology optimization on the heatmap and the part affinity field to obtain a set of matching joints. Non-maximum suppression is then used to remove redundancy from the set of matching joints to ensure the uniqueness of joints in a single frame, resulting in refined joint coordinates. Adjacent frame matching is performed on the refined joint coordinates to obtain temporally continuous joint trajectories. Kalman filtering is then used to smooth and interpolate the temporally continuous joint trajectories, removing low-confidence nodes caused by missing depth data, and outputting a sequence of skeletal node coordinates.

[0031] S1.5. Cross-modal fusion of motion frequency domain features and skeletal node coordinate sequences is performed to construct a skeletal motion feature matrix. Specifically, the motion frequency domain features are aligned to the time axis using the PTP time protocol, and the timestamps of the skeletal node coordinate sequences are matched using dynamic time warping to eliminate the temporal offset caused by the difference in sampling rates. Subsequently, a weighted fusion method is used to concatenate and cross-modal fuse the skeletal node coordinate sequences with the corresponding motion frequency domain features to form an extended feature vector. Then, the skeletal node coordinate sequences are divided into 5 limb groups according to human anatomy. For each limb group, K-nearest neighbor search is used to perform similarity matching on the extended feature vectors to obtain limb feature combinations. The limb feature combinations are then concatenated according to the physical connection order to obtain limb feature vectors. Finally, all limb feature vectors are stacked along the time axis to construct a skeletal motion feature matrix. The values ​​of each dimension of the skeletal motion feature matrix are compressed to the [0,1] interval by range normalization to eliminate dimensional differences. It should be noted that human anatomical structure refers to the limb division method defined by the International Society for Biomechanics (ISB) standard, based on the definition of human kinematics; physical connection sequence refers to the anatomical adjacency relationship between joints, based on the definition of clinical anatomical atlases.

[0032] S2. Input the skeletal motion state matrix into the skeletal motion multidimensional quantification model. The time series analysis layer performs time series segmentation and dynamic behavior classification. The action evaluation layer performs multidimensional index calculation and weight allocation, and outputs a comprehensive score for rock climbing actions.

[0033] Specifically, the operations include the following: S2.1 Construct and train a multi-dimensional quantization model for skeletal motion. Specifically, in the PyTorch framework, a bidirectional LSTM network architecture is called using the `nn.Module` parameter, and deformable convolutions are embedded into it. The input dimension is set to 512, the hidden layer dimension to 256, and the time step to 120 frames. A LayerNorm layer is then applied after the bidirectional LSTM network for feature normalization, and a Dropout layer is used for random deactivation to prevent overfitting, thus completing the temporal analysis layer. The `nn.MultiheadAttention` function is used to call the Transformer network, and a spatiotemporal attention mechanism is used for joint weight allocation. The number of heads is set to 8, the key dimension to 64, and the value dimension to 64. A fully connected layer is then applied after the Transformer network for feature projection to adapt to the scoring output dimension, and a Sigmoid activation layer is used for non-linear mapping to enhance feature representation capabilities, thus completing the action evaluation layer. Residual connections are used to stack features between the temporal analysis layer and the action evaluation layer to obtain cross-layer fusion features. Channel concatenation is then performed on the cross-layer fusion features to generate enhanced feature representations. Deformable convolutions are used to perform local geometric transformations on the enhanced feature representations to obtain spatial structure features. Based on the spatial structure features, global average pooling is used for feature compression to generate temporal aggregate features. Fully connected layer mapping and sigmoid activation are then performed on the temporal aggregate features to obtain comprehensive fusion weights. Based on the comprehensive fusion weights, the temporal analysis layer and the action evaluation layer are stacked with cross-layer parameters to complete the construction of a multi-dimensional quantization model of skeletal motion. Next, the skeletal motion multidimensional quantization model is trained. Further, the skeletal motion feature matrix is ​​divided into a sample set, a training set, and a validation set. On the sample set, Xavier initialization is used for parameter initialization, and random rotation is used to augment the samples, forming enhanced training samples. On the training set, the AdamW optimizer is used to perform gradient backpropagation on the augmented training samples, and gradient clipping is applied simultaneously for gradient norm constraints to obtain updated skeletal motion multidimensional quantization model parameters. On the validation set, the updated skeletal motion multidimensional quantization model parameters are forward-inferred using a smoothed L1 loss function to obtain the validation loss. When the validation loss exceeds the convergence threshold for 10 consecutive rounds, training terminates, and the trained skeletal motion multidimensional quantization model is output simultaneously. It should be noted that the convergence threshold is defined based on the rate of change of the first derivative of the validation loss, and its value ranges from [0.001, 0.005].

[0034] S2.2 The temporal analysis layer uses bidirectional LSTM units for time series segmentation and dynamic behavior classification, outputting the current action stage identifier. Specifically, the skeletal motion feature matrix is ​​input into the skeletal motion multidimensional quantization model through the Input interface. Dimension alignment and feature reorganization are performed on the skeletal motion feature matrix through 1×1 convolution to obtain a standardized feature sequence. A Dropout layer is applied to the standardized feature sequence for random deactivation, outputting a stable feature sequence. The forward LSTM unit performs time-step unrolling on the stable feature sequence to obtain forward temporal features. Residual connections are performed on the forward temporal features to generate enhanced forward features. The backward LSTM unit performs reverse time-step processing on the stable feature sequence to form backward temporal features. Skip connections are used to perform gradient preservation and feature fusion on the backward temporal features to obtain enhanced backward features. A 1×1 convolution is applied to perform time-series segmentation of the enhanced forward and backward features, obtaining a bidirectional feature sequence. This bidirectional feature sequence is then concatenated along its feature dimension to form a 512-dimensional fused feature. Subsequently, a fully connected layer maps the 512-dimensional fused feature to the action stage category space, and a Softmax function is used for probability normalization to generate the probability distribution for each stage. Simultaneously, a three-level dispersion threshold is used to dynamically classify the probability distribution for each stage. For example, a sliding window is used to perform probability difference quantization on the probability distribution for each stage to obtain the KL divergence. When the KL divergence is within the range of the first-level dispersion threshold... The action sequence is divided into three phases: grasping, transition, and locking. When the KL divergence is within the second-order discreteness threshold, it is classified as a transition phase; when the KL divergence is within the third-order discreteness threshold, it is classified as a locking phase. The phases are then smoothed and merged to obtain an initial action phase sequence. A Conditional Random Field (CRF) is applied to the initial action phase sequence to constrain probabilistic behavior and prevent non-physiological phase jumps, thus obtaining an optimized phase sequence. A sliding window is applied to smooth the optimized phase sequence, ensuring the temporal continuity of phase transitions, and outputting action phase identifiers, including the preparation phase, the exertion phase, and the maintenance phase. It should be noted that the three-level dispersion threshold is defined based on the statistical characteristics of the KL divergence of the probability distribution of each stage, and the value range is [0, 0.3]. Non-physiological stage jumps refer to stage transitions that do not conform to the laws of human biomechanics, such as jumping directly from the "preparation period" to the "maintenance period".

[0035] S2.3 The action evaluation layer applies a spatiotemporal attention mechanism to perform multi-dimensional index calculation and weight allocation on the current action stage identifier to obtain a four-dimensional score. Specifically, one-dimensional convolution is used to compress features, converting the action stage identifier into a 128-dimensional feature vector, and LayerNorm is applied to standardize the features to obtain normalized features. The spatiotemporal attention mechanism is then applied to perform spatiotemporal weight quantization on the normalized features. The spatial attention unit performs linear transformation and non-linear activation on the spatial sequence of the normalized features through two fully connected layers to generate a spatial attention score, and applies the Softmax function for normalization to obtain spatial weights. The temporal attention unit uses one-dimensional convolution to perform sliding window convolution on the temporal sequence of the normalized features to extract temporal patterns, and applies the Sigmoid function to perform gating adjustment on the temporal patterns to obtain temporal attention scores. Simultaneously, the Softmax function is used to normalize the temporal attention scores to obtain temporal weights. Next, spatial and temporal weights are fused using Hadamard product to generate a spatiotemporal joint attention map. This spatiotemporal joint attention map is then multiplied by a 128-dimensional feature vector to obtain weighted features. Four independent fully connected layers are used to calculate multidimensional indicators on these weighted features. The first fully connected layer performs weight multiplication and bias stacking on the weighted features using weight matrix multiplication, and applies the Swish activation function for nonlinear mapping to obtain a strength dimension score. The second fully connected layer applies cosine similarity calculation to perform joint coordination quantification and feature aggregation on the weighted features, generating a coordination dimension score. The third fully connected layer applies analysis of variance to perform volatility calculation and stability quantification on the weighted features, obtaining a stability dimension score. The fourth fully connected layer uses the energy efficiency formula to calculate the power consumption ratio and compress the features, and applies gradient pruning to limit the range, forming an efficiency dimension score. The scores for strength, coordination, stability, and efficiency are dynamically weighted and initially fused to obtain a weighted score vector. A linear combination operation is then performed on this weighted score vector using a weighted summation method to generate the four-dimensional score values. The specific mathematical formula is as follows. ; in, This represents the four-dimensional score. This indicates the weight of the strength dimension. This indicates a score representing the strength dimension. This indicates the weight of the coordination dimension. The score represents the coordination dimension. The weights representing the stability dimension. The score represents the stability dimension. The weights represent the efficiency dimension. This indicates the score for the efficiency dimension.

[0036] S2.4. Generate a comprehensive climbing action score based on the current action stage identifier and the four-dimensional score values. Specifically, a Gaussian kernel function is used to perform a non-linear feature transformation on the current action stage identifier, mapping it to a 128-dimensional stage identifier vector. Simultaneously, the four-dimensional score values ​​are uniformly scaled to 128 dimensions through a fully connected layer, forming a four-dimensional score vector to ensure dimensional matching. LayerNorm is used to standardize both the stage identifier vector and the four-dimensional score vector to eliminate dimensional differences. Then, the stage identifier vector and the four-dimensional score vector are directly concatenated along the feature channel dimension to form a 256-dimensional comprehensive feature vector. Next, a weighted summation method is used to perform weight allocation and aggregation on the comprehensive feature vector to generate an initial comprehensive score. A bias term is used to adjust the scoring benchmark of the initial comprehensive score, and Min-Max standardization is applied for numerical transformation and range limitation to output the comprehensive climbing action score. This comprehensive climbing action score integrates temporal stage information and multi-dimensional motion indicators, providing a unified quantitative benchmark for action quality assessment. It should be noted that the bias term is defined based on the baseline offset of the initial comprehensive score, and its value range is [-0.5, 0.5].

[0037] S3. Perform differential calculation on the comprehensive score and standard action score of rock climbing action to obtain the quantized offset tensor. Perform feature mapping and transformation superposition on the quantized offset tensor and skeletal motion parameters through improved B-spline surface fitting to generate a skeletal superposition comparison view.

[0038] Specifically, the operations include the following: S3.1. Perform differential calculations on the comprehensive score and standard action score of climbing movements to obtain a quantized offset tensor. Specifically, the standard action score is defined based on the IFSC's tiered zoning rules, and cubic spline interpolation is used to discretize the standard action score to obtain discrete points. Curve fitting and parameter optimization are performed on the discrete points of the standard action score using the least squares method to generate a continuous function of the standard action score. Time resampling is used to uniformly sample the continuous function of the standard action score to obtain the standard action score curve. The standard action score curve reflects the spatiotemporal score distribution of ideal climbing movements. Next, linear interpolation was used to sample and fill missing values ​​in the comprehensive score of climbing movements at equal intervals. Noise suppression and smoothing were then performed using moving average to obtain the actual movement score curve. Five integration nodes from the standard movement score curve and the actual movement score curve were extracted at equal intervals using the Gaussian integral method, and different weight coefficients were assigned to each integration node using the Gauss-Legendre formula. Based on the assigned weight coefficients, the actual movement score curve and the standard movement score curve were numerically integrated using a weighted summation method within the same time interval to obtain the area difference. The area difference was then normalized and non-negatively processed using range calculation to obtain the original offset value. The specific mathematical formula is as follows. ; in, Indicates the original offset value. This represents the total number of integration nodes. Indicates the index of the integral node. Indicates the first The weight coefficients of each integration node. Indicates a time index. This indicates the actual action scoring curve at time point. The rating value ; Next, the original offset values ​​are processed by piecewise linear quantization: the original offset values ​​are divided into four sub-intervals according to continuous intervals, and different sub-intervals use different quantization step sizes. For example, the original offset values ​​are divided into intervals of [0,0.2], [0.2,0.5], [0.5,0.8], and [0.8,1]. The [0,0.2] sub-interval uses a fine step size of 0.05, while the [0.8,1] interval uses a larger step size of 0.1. According to the quantization step size, the original offset values ​​are linearly scaled and integer mapped to obtain quantized discrete values. Tensor concatenation is performed on the quantized discrete values ​​to generate a quantized offset tensor. The quantized offset tensor reflects the degree to which the actual action deviates from the standard action, enabling a lightweight digital representation of action quality assessment.

[0039] S3.2. Multi-scale splicing of the quantized offset tensor and skeletal motion parameters is performed to obtain the spatiotemporal feature tensor. An improved B-spline surface fitting is used to perform feature mapping on the spatiotemporal feature tensor to generate the skeletal motion basis surface. In specific operations, the quantized offset tensor is dequantized by linear interpolation to restore the integer quantized offset tensor to a continuous value domain and obtain the dequantized offset value. At the same time, principal component analysis is applied to perform dimensionality reduction decomposition of the skeletal motion parameters to extract multimodal features of skeletal motion. Subsequently, the dynamic time warping (DTW) algorithm is used to perform time series alignment of the dequantized offset value and the multimodal features of skeletal motion to eliminate time axis offset and ensure motion phase consistency. Then, along the feature channel dimension, the dequantized offset value and the multimodal features of skeletal motion are spliced ​​at multiple scales to form the initial spatiotemporal feature tensor. Multi-scale energy spectrum decomposition is performed on the initial spatiotemporal feature tensor using the db4 wavelet basis to extract low-frequency global motion trends and high-frequency local detail features. Based on low-frequency global motion trends and high-frequency local detail features, an improved B-spline surface fitting method is used for feature mapping: Savitzky-Golay filtering is applied to smoothly fit the low-frequency global motion trend, generating joint motion trajectories; peak localization of the joint motion trajectory is performed using the curvature extremum detection method to extract potential control points, and density clustering of the potential control points is performed using DBSCAN clustering to generate high curvature extremum points; time-frequency decomposition and energy focusing are performed on the high-frequency local detail features using short-time Fourier transform to form a local motion detail spectrum, and the local motion details are then analyzed. Nonmaximum suppression is applied to the spectrum to obtain salient motion features. Based on the high curvature extrema and salient motion features, control points are fitted using non-uniform rational B-splines (NURBS) to obtain a surface control mesh. Weight optimization and node parameter adjustment are performed on the surface control mesh to construct surface basis functions. Local refinement and parameter fitting of the surface basis functions are performed through hierarchical B-spline iteration to obtain the skeletal motion basis surface. The degree of deformation of the skeletal motion basis surface can intuitively reflect the severity of the deviation of the actual movement from the standard movement, providing a characterization basis with both geometric intuition and numerical accuracy for subsequent analysis. It should be noted that the improved B-spline surface fitting makes the following improvements based on the existing B-spline surface fitting: on the one hand, peak location is achieved by using the curvature extremum detection method, and control points are fitted by using non-uniform rational B-splines, so as to achieve accurate capture of potential control points and smooth continuity of motion trajectory; on the other hand, local refinement is performed by using hierarchical B-splines iteratively, which improves the geometric fidelity and convergence efficiency of the skeletal motion basis surface under complex motion postures.

[0040] S3.3. The ARAP deformation method is applied to transform and superimpose the skeletal motion basis surface, and the norm smoothing operator is used for regularization constraints to form a skeletal superimposed comparison view. In specific operations, the ARAP deformation method is used to optimize vertex displacement and rigidity of the skeletal motion basis surface to obtain a deformed surface, and the deformed surface is transformed and superimposed to obtain an initial superimposed result. Then, the norm smoothing operator is used to measure the difference between adjacent transformations of the initial superimposed result to obtain a smoothing energy term. The Gauss-Newton method is used to perform regularization constraints and weight balancing on the smoothing energy term to balance the deformation degrees of freedom and smoothness of the deformed surface, and the regularized deformation result is output. Laplacian smoothing is used to perform neighborhood weighted averaging on the regularized deformation result to eliminate local distortion and generate a smooth superimposed surface. The Unity engine is used to render the deformation amplitude of the smooth superimposed surface to form a skeletal superimposed comparison view. It should be noted that the norm smoothing operator is based on the Frobenius norm difference definition of deformable surfaces, and its value range is [0.1, 0.5].

[0041] S4. Use wavelet transform to extract climbing phase features from the skeleton overlay comparison view, and perform real-time analysis of climbing actions based on the climbing phase features.

[0042] Specifically, the operations include the following: S4.1. Wavelet transform is used to perform multi-level decomposition and phase angle quantization on the skeleton overlay comparison view to extract climbing phase features. Specifically, the vertex displacement amplitude sequence in the skeleton overlay comparison view is extracted, and sliding window standardization is applied to remove dimensions and smooth the vertex displacement amplitude sequence to generate a standardized motion signal. Tensor reconstruction is used to reconstruct the spatiotemporal dimensions of the standardized motion signal to obtain a time-space signal matrix. Subsequently, the db4 wavelet basis is used to perform frequency band separation on the time-space signal matrix to obtain high-frequency, mid-frequency, and low-frequency components. The high-frequency, mid-frequency, and low-frequency components are then analyzed. The low-frequency components are decomposed into three layers. The first layer performs bandpass filtering on the high-frequency components to capture high-frequency jitter components (such as finger fine-tuning). The second layer performs energy integration on the mid-frequency components to extract mid-frequency force characteristics (such as push-off movements). The third layer performs trend fitting on the low-frequency components to obtain the overall low-frequency motion trend (such as trunk displacement). The high-frequency jitter components, mid-frequency force characteristics, and overall low-frequency motion trends are integrated to output multi-frequency motion characteristics. Hilbert transform is applied to quantize the phase angle of the multi-frequency motion characteristics to obtain the time-frequency phase distribution. The time-frequency phase distribution is then enhanced to output climbing phase characteristics.

[0043] S4.2 Based on climbing phase features, the Analytic Hierarchy Process (AHP) is used to perform real-time analysis of joint coordination and motion coordination quality in rock climbing movements. Specifically, a sliding window is used to traverse the climbing phase features, outputting a sequence of motion cycle markers. Then, the AHP is used to assign weights and prioritize the motion cycle marker sequences to obtain a comprehensive weight distribution. Eigenvalue decomposition is applied to extract and normalize the comprehensive weight distribution, resulting in a standardized weight vector. Based on the standardized weight vector, a short-time Fourier transform is used for frequency domain decomposition to obtain a phase coherence vector. Based on the phase coherence vector, the motion coordination is then analyzed. Same-sex quantification and anomaly labeling are used to complete joint coordination analysis and obtain a joint coordination degree matrix. Simultaneously, based on standardized weight vectors, fuzzy logic is used for rule reasoning and membership degree assignment to generate coordination quality levels. According to the coordination quality levels, climbing movements are graded and feedback prompts are provided to complete the analysis of movement coordination quality and generate coordination quality levels. The joint coordination degree matrix and coordination quality levels are weighted, fused, and feature-overlayed to obtain movement quantification data. The movement quantification data is time-series aligned using exponential moving average to form structured analysis results. The structured analysis results are then multi-dimensionally integrated and dynamically rendered to output a real-time analysis report.

[0044] This embodiment also provides a climbing motion analysis system based on multi-dimensional quantitative evaluation, including: a registration and fusion module, which uses the ICP algorithm combined with the OpenPose algorithm to perform spatial registration and bone node localization on a multimodal climbing dataset to form a bone motion state matrix; The multidimensional quantization module inputs the skeletal motion state matrix into the skeletal motion multidimensional quantization model, the time series analysis layer performs time series segmentation and dynamic behavior classification, and the action evaluation layer performs multidimensional index calculation and weight allocation, outputting a comprehensive score for rock climbing actions. The dynamic mapping module performs differential calculations on the comprehensive score and standard action score of rock climbing action to obtain the quantized offset tensor. Through improved B-spline surface fitting, feature mapping and transformation superposition are performed on the quantized offset tensor and skeletal motion parameters to generate a skeletal superposition comparison view. The real-time analysis module uses wavelet transform to extract climbing phase features from the skeleton overlay comparison view, and performs real-time analysis of climbing actions based on the climbing phase features, outputting a real-time analysis report.

[0045] This embodiment also provides a computer device applicable to the rock climbing motion analysis method based on multi-dimensional quantitative evaluation, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the rock climbing motion analysis method based on multi-dimensional quantitative evaluation as proposed in the above embodiment.

[0046] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0047] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the climbing motion analysis method based on multi-dimensional quantitative evaluation as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0048] In summary, this invention significantly improves the spatial registration accuracy of multimodal climbing data by fusing the ICP algorithm and the OpenPose algorithm, ensuring the accuracy and stability of skeletal node localization. Simultaneously, it constructs a multidimensional quantification model of skeletal motion that includes a temporal analysis layer and a motion evaluation layer, enabling refined dynamic classification and multidimensional index evaluation of climbing movements, thereby effectively improving the overall accuracy and scientific rigor of motion analysis.

[0049] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A climbing motion analysis method based on multi-dimensional quantitative evaluation, characterized in that: include, The ICP algorithm combined with the OpenPose algorithm was used to perform spatial registration and skeletal node localization on the multimodal climbing dataset, forming a skeletal motion state matrix; The skeletal motion state matrix is ​​input into the skeletal motion multidimensional quantification model. The time series analysis layer performs time series segmentation and dynamic behavior classification, and the action evaluation layer performs multidimensional index calculation and weight allocation to output a comprehensive score of rock climbing action. The comprehensive score and standard score of rock climbing action are differentially calculated to obtain the quantized offset tensor. The quantized offset tensor and skeletal motion parameters are then subjected to feature mapping and transformation superposition through improved B-spline surface fitting to generate a skeletal superposition comparison view. Wavelet transform is used to extract climbing phase features from the skeleton overlay comparison view, and climbing actions are analyzed in real time based on the climbing phase features, and a real-time analysis report is output.

2. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 1, characterized in that: The multimodal climbing dataset includes raw RGB-D climbing data, skeletal motion parameters, and rock wall environmental parameters.

3. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 2, characterized in that: The process of obtaining the skeletal motion feature matrix specifically includes the following steps. The ICP algorithm is applied to spatially register the multimodal climbing dataset to obtain a spatiotemporally synchronized dataset; temporal domain analysis is performed on the spatiotemporally synchronized dataset to form motion frequency domain features; The OpenPose algorithm was applied to perform skeletal node localization on a spatiotemporal synchronization dataset to obtain a sequence of skeletal node coordinates. The motion frequency domain features and the skeletal node coordinate sequence are fused across modes to construct a skeletal motion feature matrix.

4. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 1, characterized in that: The output climbing action comprehensive score specifically includes the following steps. A temporal analysis layer and a motion evaluation layer are constructed, and deformable convolution and residual connections are applied for parameterized stacking to build a multidimensional quantitative model of skeletal motion. The skeletal motion feature matrix is ​​input into the skeletal motion multidimensional quantization model. The time series analysis layer uses bidirectional LSTM units to perform time series segmentation and dynamic behavior classification, and outputs the current action stage identifier. The action evaluation layer uses a spatiotemporal attention mechanism to perform multi-dimensional index calculation and weight allocation on the current action stage identifier to obtain a four-dimensional score value; The current action stage identifier and the four-dimensional score value are concatenated in the feature channel to generate a comprehensive feature vector; The weighted summation method is used to perform weight allocation and aggregation on the comprehensive feature vector to form a comprehensive score for rock climbing actions.

5. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 1, characterized in that: The process of obtaining the quantization offset tensor specifically includes the following steps. The climbing action comprehensive score and standard action score are differentially calculated by Gaussian integral, and the range is normalized by Min-Max standardization to generate the original offset value. The original offset value is piecewise linearly quantized to obtain the quantized offset tensor; The standard action scoring is based on a tiered zoning rule definition.

6. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 5, characterized in that: The process of generating the overlay comparison view of the skeleton includes the following steps. Multi-scale stitching of the quantized offset tensor and skeletal motion parameters is performed to obtain the spatiotemporal feature tensor; An improved B-spline surface fitting method is used to perform feature mapping on the spatiotemporal feature tensor to generate the skeletal motion basis surface; The ARAP deformation method is applied to transform and superimpose the basic surface of the skeleton motion, and the norm smoothing operator is used for regularization constraints to form a skeleton superimposed comparison view.

7. The climbing motion analysis method based on multi-dimensional quantitative evaluation as described in claim 1, characterized in that: The real-time analysis of climbing movements based on climbing phase characteristics specifically includes the following steps. Wavelet transform is used to perform multi-level decomposition and phase angle quantization on the superimposed skeleton comparison view to extract climbing phase features; Based on climbing phase characteristics, the analytic hierarchy process (AHP) is used to perform real-time analysis of joint coordination and motion coordination quality in rock climbing movements, generate quantitative motion data, integrate the quantitative motion data, and output a real-time analysis report.

8. A climbing motion analysis system based on multi-dimensional quantitative evaluation, based on the climbing motion analysis method based on multi-dimensional quantitative evaluation as described in any one of claims 1 to 7, characterized in that: include, The registration and fusion module uses the ICP algorithm combined with the OpenPose algorithm to perform spatial registration and skeletal node localization on the multimodal climbing dataset, forming a skeletal motion state matrix. The multidimensional quantization module inputs the skeletal motion state matrix into the skeletal motion multidimensional quantization model, the time series analysis layer performs time series segmentation and dynamic behavior classification, and the action evaluation layer performs multidimensional index calculation and weight allocation, outputting a comprehensive score for rock climbing actions. The dynamic mapping module performs differential calculations on the comprehensive score and standard action score of rock climbing action to obtain the quantized offset tensor. Through improved B-spline surface fitting, feature mapping and transformation superposition are performed on the quantized offset tensor and skeletal motion parameters to generate a skeletal superposition comparison view. The real-time analysis module uses wavelet transform to extract climbing phase features from the skeleton overlay comparison view, and performs real-time analysis of climbing actions based on the climbing phase features, outputting a real-time analysis report.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the climbing motion analysis method based on multi-dimensional quantitative evaluation as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the climbing motion analysis method based on multi-dimensional quantitative evaluation as described in any one of claims 1 to 7.

Images