Intelligent skiing teaching system based on VR

Abstract

The invention relates to the field of VR skiing, in particular to an intelligent skiing teaching system based on VR, and the system comprises a physical conversion module which carries out the physical feature conversion, so as to generate a foot COP, a waist inclination angle and a trunk stability index; the gravity center feature extraction module generates gravity center space-time features through a gravity center feature extraction model based on a space-time fusion convolution architecture; the gravity center feature fusion module generates global gravity center prediction features through a feature fusion model based on an LSTM network and an attention mechanism architecture; the center-of-gravity position prediction module generates a predicted center-of-gravity position through a convolutional full-connection layer based on dynamic center-of-mass guidance; and the adjusting module is used for adjusting the action of the VR teaching and skiing simulator based on the predicted gravity center position. According to the method, multi-dimensional and more comprehensive capture of key dynamic features of the skiing posture is realized, the precision and real-time performance of center-of-gravity prediction are realized, and a reliable decision basis is provided for intelligent teaching of VR skiing sports.

Description

Technical Field

[0001] This invention relates to the field of VR skiing, and more particularly to a VR-based intelligent teaching system for skiing. Background Technology

[0002] VR skiing intelligent teaching devices rely on head-mounted displays and ski simulators, along with sensors for the feet, waist, and hands. Through a virtual host and simulated character models in a virtual environment, the device processes and feeds back information such as positional and pressure changes during training to the head-mounted display, creating a sense of presence. The virtual host matches training movements, allowing trainees to correct their techniques and improve their skiing skills through simulation. Existing tests of VR skiing intelligent teaching devices show that training with VR intelligent snowboarding devices gradually builds beginners' confidence, strengthens their awareness of safety and protection, and avoids subjective exercise fatigue. Simultaneously, through real-time computer animation and interactive feedback via visual and other senses, users can not only clearly identify and correct errors in their techniques but also quickly discover changes in force application and body movement patterns during the technical process, enabling targeted training.

[0003] While existing VR skiing intelligent teaching devices have made progress in scene reproduction, they still fall short in terms of motion capture accuracy. Current devices only collect limb movement trajectories through visual sensors, neglecting the quantitative analysis of the crucial center of gravity position representing skiing posture. Furthermore, sensor data processing often employs single feature extraction methods, failing to effectively integrate dynamic changes in the spatiotemporal dimensions, resulting in low center of gravity prediction accuracy and significant lag.

[0004] Therefore, how to predict dynamic changes in the center of gravity in real time and adaptively adjust teaching strategies accordingly in VR skiing intelligent teaching, so as to adjust the center of gravity position in time when the user is about to shift their center of gravity, and improve the scientificity and effectiveness of VR skiing teaching, is a technical problem that needs to be solved. Summary of the Invention

[0005] To this end, the present invention provides a VR-based intelligent teaching system for skiing. By converting physical features of sensor data from multiple parts of the body, such as the feet, waist, and hands, and combining a feature fusion model with a spatiotemporal fusion convolutional architecture, LSTM, and attention mechanism, the system achieves multi-dimensional and more comprehensive capture of key dynamic features of skiing posture, and realizes the accuracy and real-time performance of center of gravity prediction, thus providing a reliable decision-making basis for VR intelligent teaching of skiing.

[0006] To achieve the above objectives, this invention proposes a VR-based intelligent teaching system for skiing, comprising:

[0007] The physical conversion module is used to convert the pressure data collected by the foot sensor, the posture acceleration data collected by the waist sensor, and the waving direction data collected by the hand sensor into physical features to generate foot COP, waist tilt angle, and trunk stability index.

[0008] The center of gravity feature extraction module is used to extract the foot COP, the waist tilt angle and the trunk stability index through a center of gravity feature extraction model based on a spatiotemporal fusion convolutional architecture to generate spatiotemporal features of the center of gravity.

[0009] The center of gravity feature fusion module is used to generate global center of gravity prediction features by using the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle and the trunk stability index through a feature fusion model based on an LSTM network and attention mechanism architecture.

[0010] The centroid position prediction module is used to pass the global centroid prediction features through a convolutional fully connected layer based on dynamic centroid guidance to generate the predicted centroid position.

[0011] An adjustment module is used to adjust the movements of the VR teaching and ski simulator based on the predicted center of gravity position.

[0012] Furthermore, the centroid feature fusion module includes:

[0013] The LSTM temporal modeling unit is used to model the spatiotemporal features of the centroid through the LSTM network in the time dimension to generate a comprehensive hidden state.

[0014] The data augmentation unit is used to perform weighted calculations on the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle, and the trunk stability index to generate enhanced attention input data.

[0015] An attention mechanism computation unit is used to process the enhanced attention input data through an attention mechanism to generate attention fusion weights;

[0016] The fusion unit is used to multiply the integrated hidden state and the attention fusion weights to generate the global centroid prediction features.

[0017] Furthermore, the enhanced attention input data includes enhanced key vectors and enhanced query vectors, and the data augmentation unit includes:

[0018] An initial vector generation subunit is used to map the foot COP, waist tilt angle and trunk stability index through multiple first training weight matrices to generate an initial key vector and an initial query vector.

[0019] An enhanced key vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial key vector using a second training weight matrix to generate an enhanced key vector.

[0020] An enhanced query vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial query vector using a third training weight matrix to generate an enhanced query vector.

[0021] Furthermore, the enhancement key vector includes a foot enhancement key vector and a waist enhancement key vector, the attention fusion weight includes a foot attention fusion weight and a waist attention fusion weight, and the attention mechanism calculation unit includes:

[0022] The foot attention calculation subunit is used to generate foot attention fusion weights by passing the enhanced query vector and the foot enhanced key vector through a first attention mechanism.

[0023] The waist attention calculation subunit is used to generate waist attention fusion weights by passing the enhanced query vector and the waist enhanced key vector through a second attention mechanism.

[0024] The fusion subunit is used to perform a first weighted summation of the foot attention fusion weight, foot value vector, foot value vector, waist attention fusion weight, and waist value vector, and then perform a second weighted summation with the trunk stability index to generate the global center of gravity prediction feature.

[0025] In particular, the LSTM temporal modeling unit performs in-depth temporal modeling of the center of gravity spatiotemporal features. Combined with an attention mechanism, it assigns targeted weights to the enhanced query vectors for key areas such as the feet and waist, achieving accurate capture of key temporal features during the dynamic changes of skiing movements. This reduces the oversensitivity of traditional models to instantaneous movement fluctuations and improves the stability of feature fusion. The data augmentation unit generates enhanced key vectors and query vectors through multi-trained weight matrix mapping and weighted operations on the pooled compressed center of gravity spatiotemporal features. This effectively integrates the correlation between foot COP, waist tilt angle, and other high-level spatiotemporal features, enhancing the representational ability of the input data and reducing the impact of noise interference on feature extraction.

[0026] Furthermore, the center of gravity position prediction module includes:

[0027] A mapping unit is used to pass the global centroid prediction features through a fully connected layer to generate initial mapping features;

[0028] The dynamic center of mass calculation unit is used to calculate the dynamic center of mass distance based on the pressure data, foot speed collected by the foot sensor, waist speed collected by the waist sensor, hand speed collected by the hand sensor, and the set weight.

[0029] The dynamic weight calculation unit is used to map the concatenated vector of the global center of gravity prediction features, foot velocity variance, and waist velocity variance to generate dynamic weights.

[0030] The weighted correction unit is used to perform weighted calculations on the initial mapping features and the dynamic centroid distance based on the dynamic weights to generate the predicted centroid position.

[0031] Furthermore, the dynamic centroid calculation unit includes:

[0032] An initial centroid calculation subunit is used to generate an initial centroid distance based on the pressure data and the set weight using a mass-weighted average method.

[0033] The center of mass offset velocity calculation subunit is used to generate the center of mass offset velocity based on the foot velocity, the waist velocity, the hand velocity, and the set weight using a velocity-weighted average method.

[0034] The summation calculation subunit is used to calculate the dynamic centroid distance based on the initial centroid distance and the centroid offset velocity.

[0035] Furthermore, the centroid feature extraction module includes:

[0036] The convolutional temporal feature extraction unit is used to extract temporal features from the foot COP, waist tilt angle and trunk stability index through a one-dimensional convolutional layer to generate foot temporal features, waist temporal features and trunk stability temporal features.

[0037] The spatial interaction unit is used to take the maximum value of the time dimension of the foot temporal features, the waist temporal features and the trunk stability temporal features respectively, and then perform feature splicing and mapping to generate the center of gravity spatiotemporal features.

[0038] In particular, by combining pressure data, multi-body velocities (foot / waist / hands), and a set weight through a dynamic centroid calculation unit, dynamic centroid distance is generated through the fusion of mass-weighted averaging and velocity-weighted averaging, making the prediction results more consistent with the physical laws of human movement. Convolutional temporal feature extraction captures the local correlation of data from various sensors in the time dimension, and highlights the spatial feature correlation of key action moments through temporal dimension maximum pooling and feature mapping.

[0039] Furthermore, the physical conversion module includes:

[0040] A foot COP generation unit is used to calculate the foot COP based on the pressure data and pressure horizontal coordinates;

[0041] The waist tilt angle generation unit is used to generate the waist tilt angle based on the arctangent function of the attitude data in the horizontal direction and the attitude data in the vertical direction;

[0042] The trunk stability index generation unit is used to calculate the trunk stability index based on the standard deviation of the center position of the foot COP, the average value of the center position of the foot COP, and the rate of change of the waist tilt angle.

[0043] Furthermore, the VR-based intelligent teaching system for skiing also includes:

[0044] The loss function construction module is used to construct a collaborative loss term based on regularization loss, which includes spatiotemporal fusion convolution parameters, LSTM network parameters, and attention mechanism parameters. It also constructs a centroid prediction loss term based on the Huber function. Based on the collaborative loss term and the centroid prediction loss term, a comprehensive loss function is constructed for collaborative training and optimization of the centroid feature extraction module, the centroid feature fusion module, and the centroid position prediction module.

[0045] Furthermore, the adjustment module includes:

[0046] The VR teaching early warning unit is used to adjust the early warning direction of VR teaching based on the predicted center of gravity position.

[0047] The ski simulator angle adjustment unit is used to adjust the ski simulator angle based on the predicted center of gravity position using the PD algorithm.

[0048] In particular, it improves the accuracy of the conversion from raw data to physical features, achieving a precise mapping from raw sensor data to physical features, providing higher quality input for subsequent feature extraction, and reducing the interference of feature noise on model prediction.

[0049] Compared with existing technologies, this invention achieves multi-dimensional and more comprehensive capture of key dynamic features of skiing posture by transforming physical features of sensor data from multiple parts of the body, such as feet, waist, and hands, and combining a feature fusion model with spatiotemporal fusion convolutional architecture, LSTM, and attention mechanism. This enables accurate and real-time prediction of the center of gravity, providing a reliable decision-making basis for intelligent teaching of VR skiing.

[0050] In particular, this invention utilizes an LSTM temporal modeling unit to perform deep temporal modeling of the center of gravity spatiotemporal features. Combined with an attention mechanism, it assigns targeted weights to the enhanced query vectors for key areas such as the feet and waist, achieving accurate capture of key temporal features during the dynamic changes of skiing movements. This reduces the oversensitivity of traditional models to instantaneous movement fluctuations and improves the stability of feature fusion. The data augmentation unit generates enhanced key vectors and query vectors through multi-trained weight matrix mapping and weighted operations on the pooled compressed center of gravity spatiotemporal features. This effectively integrates the correlation between foot COP, waist tilt angle, and other high-level spatiotemporal features, enhancing the representational ability of the input data and reducing the impact of noise interference on feature extraction.

[0051] In particular, this invention combines pressure data, multi-body velocities (foot / waist / hand), and a set weight through a dynamic centroid calculation unit. It then generates a dynamic centroid distance by fusing mass-weighted averaging and velocity-weighted averaging, making the prediction results more closely match the physical laws of human movement. Convolutional temporal feature extraction captures the local correlations of various sensor data in the temporal dimension, and highlights the spatial feature correlations of key action moments through temporal dimension maximum pooling and feature mapping. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the structure of a VR-based intelligent teaching system for skiing, according to an embodiment of the present invention.

[0053] Figure 2 This is a flowchart illustrating the center-of-gravity feature fusion module of the VR-based intelligent skiing teaching system according to an embodiment of the present invention.

[0054] Figure 3 This is a flowchart illustrating the center of gravity position prediction module of a VR-based intelligent teaching system for skiing, according to an embodiment of the present invention.

[0055] Figure 4 This is a flowchart illustrating the loss function construction module of the VR-based intelligent teaching system for skiing, according to an embodiment of the present invention. Detailed Implementation

[0056] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0057] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0059] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0060] like Figures 1 to 4 As shown, this invention provides a VR-based intelligent teaching system for skiing. By converting physical features from sensor data of multiple body parts such as feet, waist, and hands, and combining a feature fusion model with spatiotemporal fusion convolutional architecture, LSTM, and attention mechanism, it achieves multi-dimensional and more comprehensive capture of key dynamic features of skiing posture, and realizes the accuracy and real-time performance of center of gravity prediction, providing a reliable decision-making basis for VR intelligent teaching of skiing.

[0061] like Figure 1 As shown, this embodiment proposes a VR-based intelligent teaching system for skiing, including:

[0062] The physical conversion module is used to convert the pressure data collected by the foot sensor, the posture acceleration data collected by the waist sensor, and the waving direction data collected by the hand sensor into physical features to generate foot COP, waist tilt angle, and trunk stability index.

[0063] The center of gravity feature extraction module is used to extract the foot COP, the waist tilt angle and the trunk stability index through a center of gravity feature extraction model based on a spatiotemporal fusion convolutional architecture to generate spatiotemporal features of the center of gravity.

[0064] The center of gravity feature fusion module is used to generate global center of gravity prediction features by using the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle and the trunk stability index through a feature fusion model based on an LSTM network and attention mechanism architecture.

[0065] The centroid position prediction module is used to pass the global centroid prediction features through a convolutional fully connected layer based on dynamic centroid guidance to generate the predicted centroid position.

[0066] An adjustment module is used to adjust the movements of the VR teaching and ski simulator based on the predicted center of gravity position.

[0067] In this embodiment, as Figure 2 As shown, the centroid feature fusion module includes:

[0068] The LSTM temporal modeling unit is used to model the spatiotemporal features of the centroid through the LSTM network in the time dimension to generate a comprehensive hidden state.

[0069] The data augmentation unit is used to perform weighted calculations on the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle, and the trunk stability index to generate enhanced attention input data.

[0070] An attention mechanism computation unit is used to process the enhanced attention input data through an attention mechanism to generate attention fusion weights;

[0071] The fusion unit is used to multiply the integrated hidden state and the attention fusion weights to generate the global centroid prediction features.

[0072] Specifically, by using the spatiotemporal features of the center of gravity as the input vector of the LSTM network, and through forget gate, input gate, output gate and state update, a comprehensive hidden state is generated as the output, so as to realize the time dependence of skiing action through LSTM, since the change of the center of gravity has obvious temporal dependence features.

[0073] Specifically, the core structural parameters of the LSTM network include: the output feature dimension of the integrated hidden state is 48, the hidden state dimension is 46, the number of stacked layers is 2, the inter-layer dropout is 0.2, the recurrent_dropout is 0.2, and the initial value of the forget gate bias is 1. The temporal attention layer dimension of the attention mechanism is 32.

[0074] Specifically, the fully connected convolutional layer sequentially comprises multi-scale temporal convolution, spatial graph convolution, and a fully connected layer. The feature dimension of the global centroid prediction feature input to the multi-scale temporal convolution is 64. The multi-scale temporal convolution includes a short-term convolutional kernel with a kernel size of 3, a medium-term convolutional kernel with a kernel size of 7, and a long-term convolutional kernel with a kernel size of 15. Each convolutional kernel outputs a channel with a dimension of 16, a stride of 1, and a ReLU activation function. The keypoint feature dimension of the spatial graph convolution is 32, and the activation function is ReLU. The output dimension of the GCN layer is [32, 64]. The fully connected layer has 64 hidden units and a Dropout value of 0.5.

[0075] In this embodiment, as Figure 2 As shown, the data enhancement unit includes:

[0076] An initial vector generation subunit is used to map the foot COP, waist tilt angle and trunk stability index through multiple first training weight matrices to generate an initial key vector and an initial query vector.

[0077] An enhanced key vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial key vector using a second training weight matrix to generate an enhanced key vector.

[0078] An enhanced query vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial query vector using a third training weight matrix to generate an enhanced query vector.

[0079] The enhanced attention input data includes the enhanced key vector and the enhanced query vector.

[0080] Specifically, the process of generating the initial key vector and the initial query vector can be represented as follows:

[0081]

[0082] In the formula, Q and K foot K waist V foot V waist W represents the initial query vector, the initial key vector belonging to foot data, the initial key vector belonging to waist data, the foot value vector, and the waist value vector, respectively. q , These represent the learnable fourth training weight matrix, the first training weight matrix belonging to foot data, the first training weight matrix belonging to waist data, the first training weight matrix belonging to foot data, and the first training weight matrix belonging to waist data, respectively. These represent the trunk stability index, foot COP, and lumbar inclination angle, respectively.

[0083] Specifically, the process of generating the enhanced key vector and the enhanced query vector can be represented as follows:

[0084]

[0085] In the formula, Q′ and K′ foot K′ waist Let Q and K represent the augmented query vector, foot augmented key vector, and waist augmented key vector, respectively. foot K waist These represent the initial query vector, the initial key vector belonging to the foot data, and the initial key vector belonging to the waist data, respectively. These represent the third training weight matrix, the second training weight matrix belonging to the foot data, and the second training weight matrix belonging to the waist data, respectively.

[0086] In this embodiment, the attention mechanism calculation unit includes:

[0087] The foot attention calculation subunit is used to generate foot attention fusion weights by passing the enhanced query vector and the foot enhanced key vector through a first attention mechanism.

[0088] The waist attention calculation subunit is used to generate waist attention fusion weights by passing the enhanced query vector and the waist enhanced key vector through a second attention mechanism.

[0089] The fusion subunit is used to perform a first weighted summation of the foot attention fusion weight, foot value vector, foot value vector, waist attention fusion weight, and waist value vector, and then perform a second weighted summation with the trunk stability index to generate the global center of gravity prediction feature.

[0090] The enhancement key vectors include foot enhancement key vectors and waist enhancement key vectors, and the attention fusion weights include foot attention fusion weights and waist attention fusion weights.

[0091] Specifically, the process of generating foot attention fusion weights and waist attention fusion weights can be represented as follows:

[0092]

[0093] In the formula, Attn foot Attn waist These represent the foot attention fusion weights and waist attention fusion weights, respectively. `softmax` represents the softmax function. Q′, d represents the transpose of the augmented query vector, the foot augmented key vector, and the waist augmented key vector, respectively. k This represents the dimension of the key vector, which is 64 in both attention mechanisms.

[0094] Specifically, the process of generating global centroid prediction features can be represented as:

[0095]

[0096] In the formula, γ represents the global centroid prediction feature, and γ represents the learnable fusion weighting parameter. Indicates the waist inclination angle, Attn foot Attn waist V represents the foot attention fusion weight and the waist attention fusion weight, respectively. foot Vwaist These represent the foot value vector and the waist value vector, respectively. Therefore, through the aforementioned attention mechanism, the model assigns higher weights to the entry and exit points of a turn during cornering maneuvers.

[0097] In particular, the LSTM temporal modeling unit performs in-depth temporal modeling of the center of gravity spatiotemporal features. Combined with an attention mechanism, it assigns targeted weights to the enhanced query vectors for key areas such as the feet and waist, achieving accurate capture of key temporal features during the dynamic changes of skiing movements. This reduces the oversensitivity of traditional models to instantaneous movement fluctuations and improves the stability of feature fusion. The data augmentation unit generates enhanced key vectors and query vectors through multi-trained weight matrix mapping and weighted operations on the pooled compressed center of gravity spatiotemporal features. This effectively integrates the correlation between foot COP, waist tilt angle, and other high-level spatiotemporal features, enhancing the representational ability of the input data and reducing the impact of noise interference on feature extraction.

[0098] In this embodiment, as Figure 3 As shown, the center of gravity position prediction module includes:

[0099] A mapping unit is used to pass the global centroid prediction features through a fully connected layer to generate initial mapping features;

[0100] The dynamic center of mass calculation unit is used to calculate the dynamic center of mass distance based on the pressure data, foot speed collected by the foot sensor, waist speed collected by the waist sensor, hand speed collected by the hand sensor, and the set weight.

[0101] The dynamic weight calculation unit is used to map the concatenated vector of the global center of gravity prediction features, foot velocity variance, and waist velocity variance to generate dynamic weights.

[0102] The weighted correction unit is used to perform weighted calculations on the initial mapping features and the dynamic centroid distance based on the dynamic weights to generate the predicted centroid position.

[0103] Specifically, the process of generating the initial mapping features can be represented as:

[0104]

[0105] In the formula, W represents the initial mapping feature. p b p Let represent the learnable weight matrix and learnable bias term used to generate the initial mapped features, respectively. This represents the global centroid prediction feature.

[0106] Specifically, the process of generating dynamic weights can be represented as:

[0107]

[0108] In the formula, λ represents the dynamic weight, σ represents the sigmoid function, Concat represents vector concatenation, and W c b c These represent the learnable weight matrix and learnable bias term used to generate dynamic weights, respectively. Var(F) represents the global center of gravity prediction feature, Var(W) represents the foot velocity variance within a set time window, and Var(W) represents the waist velocity variance within a set time window.

[0109] Specifically, the process of generating the predicted centroid position can be represented as:

[0110]

[0111] In the formula, Indicates the predicted center of gravity location. Let λ represent the initial mapping features, and λ represent the dynamic weights. This represents the dynamic centroid distance.

[0112] In this embodiment, as Figure 3 As shown, the dynamic centroid calculation unit includes:

[0113] An initial centroid calculation subunit is used to generate an initial centroid distance based on the pressure data and the set weight using a mass-weighted average method.

[0114] The center of mass offset velocity calculation subunit is used to generate the center of mass offset velocity based on the foot velocity, the waist velocity, the hand velocity, and the set weight using a velocity-weighted average method.

[0115] The summation calculation subunit is used to calculate the dynamic centroid distance based on the initial centroid distance and the centroid offset velocity.

[0116] Specifically, the process of generating the initial centroid distance can be represented as:

[0117]

[0118] In the formula, The initial centroid distance, m f m w m h Let P represent the mass distribution of the feet, waist, and hands, respectively. f P w P h These represent the foot position coordinates (y-direction), torso position coordinates (y-direction), and arm position coordinates (y-direction) calculated from pressure data, respectively. Specifically, the process of generating the center of mass offset velocity can be represented as:

[0119]

[0120] In the formula, v com The velocity of the centroid shift is represented by m. f m w m h These represent the mass distributions of the feet, waist, and hands, respectively. f v w v h These represent the foot speed, waist speed, and hand speed within the set time window, respectively.

[0121] The foot, waist, and hand weight distributions are 0.7, 0.45, and 0.11 of the set body weight, respectively, which conforms to the standard human body weight distribution.

[0122] Specifically, the dynamic centroid distance is obtained by multiplying the centroid offset velocity by the duration of a set time window and then adding the initial centroid distance. The set time window is preferably 1 second.

[0123] In this embodiment, the centroid feature extraction module includes:

[0124] The convolutional temporal feature extraction unit is used to extract temporal features from the foot COP, waist tilt angle and trunk stability index through a one-dimensional convolutional layer to generate foot temporal features, waist temporal features and trunk stability temporal features.

[0125] The spatial interaction unit is used to take the maximum value of the time dimension of the foot temporal features, the waist temporal features and the trunk stability temporal features respectively, and then perform feature splicing and mapping to generate the center of gravity spatiotemporal features.

[0126] Specifically, the process of generating temporal features of the feet, waist, and trunk can be represented as follows:

[0127]

[0128] In the formula, These represent the temporal features of the feet, waist, and trunk, respectively, and Conv1D represents a one-dimensional convolutional layer. θ represents the foot COP, the waist tilt angle, and the trunk stability index, respectively. f θ w θ d This represents the parameters of the three convolutional layers.

[0129] Specifically, the process of generating the spatiotemporal features of the centroid can be represented as:

[0130]

[0131] In the formula, Represents the temporal features of the feet, waist, and trunk, respectively; ReLU represents the ReLU activation function; W s b s Let represent the learnable weight matrix and learnable bias term for generating the spatiotemporal features of the centroid, respectively. Concat represents feature concatenation. This indicates that the temporal features of the foot are taken as the maximum value in the time dimension. This indicates that the maximum value of the time dimension of the waist temporal features is taken. This indicates that the stable temporal features of the torso are taken as the maximum value in the time dimension.

[0132] In particular, by combining pressure data, multi-body velocities (foot / waist / hands), and a set weight through a dynamic centroid calculation unit, dynamic centroid distance is generated through the fusion of mass-weighted averaging and velocity-weighted averaging, making the prediction results more consistent with the physical laws of human movement. Convolutional temporal feature extraction captures the local correlation of data from various sensors in the time dimension, and highlights the spatial feature correlation of key action moments through temporal dimension maximum pooling and feature mapping.

[0133] In this embodiment, the physical conversion module includes:

[0134] A foot COP generation unit is used to calculate the foot COP based on the pressure data and pressure horizontal coordinates;

[0135] The waist tilt angle generation unit is used to generate the waist tilt angle based on the arctangent function of the attitude data in the horizontal direction and the attitude data in the vertical direction;

[0136] The trunk stability index generation unit is used to calculate the trunk stability index based on the standard deviation of the center position of the foot COP, the average value of the center position of the foot COP, and the rate of change of the waist tilt angle.

[0137] Specifically, the process of generating foot COP can be represented as:

[0138]

[0139] In the formula, COP x Indicates COP,F of the foot i x i Let ∑F represent the vertical force and horizontal position coordinates of the i-th foot pressure sensor. j This represents the total pressure from all foot pressure sensors.

[0140] Specifically, the process of generating the waist inclination angle can be represented as:

[0141]

[0142] In the formula, Indicates the waist inclination angle, tan -1 Let a represent the arctangent function. y a z These represent the vertical and horizontal attitude data of the waist IMU sensor, respectively.

[0143] Specifically, the process of generating the trunk stability index can be represented as:

[0144]

[0145] In the formula, The trunk stability index, σ(COP) x ) represents the standard deviation of the center position of the foot COP for all foot pressure sensors, μ(COP) x This represents the average center-of-gravity position of the foot COP across all foot pressure sensors. This indicates the rate of change of the waist inclination angle.

[0146] In this embodiment, as Figure 4 As shown, the VR-based intelligent teaching system for skiing also includes:

[0147] The loss function construction module is used to construct a collaborative loss term based on regularization loss, which includes spatiotemporal fusion convolution parameters, LSTM network parameters, and attention mechanism parameters. It also constructs a centroid prediction loss term based on the Huber function. Based on the collaborative loss term and the centroid prediction loss term, a comprehensive loss function is constructed for collaborative training and optimization of the centroid feature extraction module, the centroid feature fusion module, and the centroid position prediction module.

[0148] Specifically, the comprehensive loss function can be expressed as:

[0149]

[0150] In the formula, Let represent the comprehensive loss function, the collaborative loss term, and the centroid prediction loss term, respectively; α and β represent the centroid prediction main loss and the regularization loss, preferably 0.9 and 0.1, respectively; and N represents the total number of samples. This represents the true centroid position (x-axis) of the i-th sample calculated using the Huber function. The predicted centroid position (x-axis) of the i-th sample. i The loss value, λ conv , λ attn , λ lstmΘ represents the regularization strength coefficients of the spatiotemporal fusion convolutional architecture, the attention mechanism, and the LSTM network, respectively. conv Θ attn Θ lstm These represent the model parameters for the spatiotemporal fusion convolutional architecture, the attention mechanism, and the LSTM network, respectively.

[0151] In this embodiment, the adjustment module includes:

[0152] The VR teaching early warning unit is used to adjust the early warning direction of VR teaching based on the predicted center of gravity position.

[0153] The ski simulator angle adjustment unit is used to adjust the ski simulator angle based on the predicted center of gravity position using the PD algorithm.

[0154] Specifically, based on the comparison between the predicted center of gravity position and the position threshold, when it is determined that the user's skiing movement is not forward enough, the VR teaching device displays the offset by a red forward arrow; when it is determined that the user's skiing movement is backward, the VR teaching device displays a yellow warning circle; and when it is determined that the user's skiing movement is lateral, the VR teaching device displays a balance guide line.

[0155] Specifically, the ski simulator angle adjustment unit adjusts the ski simulator angle by using the difference between the predicted center of gravity position and the current center of gravity position as an error through the PD algorithm of the PID control algorithm, wherein the P control parameter is preferably 0.8 and the D control parameter is preferably 0.2.

[0156] In particular, it improves the accuracy of the conversion from raw data to physical features, achieving a precise mapping from raw sensor data to physical features, providing higher quality input for subsequent feature extraction, and reducing the interference of feature noise on model prediction.

[0157] In this embodiment, by transforming physical features from sensor data from multiple body parts (foot, waist, and hands), and combining a feature fusion model with a spatiotemporal fusion convolutional architecture, LSTM, and attention mechanisms, a multi-dimensional and more comprehensive capture of key dynamic features of skiing postures is achieved. This enables accurate and real-time prediction of the center of gravity, providing a reliable decision-making foundation for intelligent VR skiing instruction. The LSTM temporal modeling unit performs in-depth temporal modeling of the center of gravity spatiotemporal features, and the attention mechanism is used to assign targeted weights to the enhanced query vectors of key body parts (foot and waist). This achieves accurate capture of key temporal features during the dynamic changes of skiing movements, reduces the oversensitivity of traditional models to instantaneous movement fluctuations, and improves the stability of feature fusion. The data augmentation unit generates enhanced key vectors and query vectors through multi-trained weight matrix mapping and weighted operations on the pooled compressed spatiotemporal features of the center of gravity. This effectively integrates the correlation between foot COP, waist tilt angle, and other high-level spatiotemporal features, improving the representational ability of the input data and reducing the impact of noise interference on feature extraction. By combining pressure data, multi-body velocities (foot / waist / hand), and a set weight using a dynamic centroid calculation unit, dynamic centroid distance is generated through the fusion of mass-weighted averaging and velocity-weighted averaging, making the prediction results more consistent with the physical laws of human movement. Convolutional temporal feature extraction captures the local correlation of data from various sensors in the temporal dimension, and highlights the spatial feature correlation of key action moments through temporal dimension max pooling and feature mapping.

[0158] Those skilled in the art will recognize that the modules and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0160] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A VR-based intelligent teaching system for skiing, characterized in that, include: The physical conversion module is used to convert the pressure data collected by the foot sensor, the posture acceleration data collected by the waist sensor, and the waving direction data collected by the hand sensor into physical features to generate foot COP, waist tilt angle, and trunk stability index. The center of gravity feature extraction module is used to extract the foot COP, the waist tilt angle and the trunk stability index through a center of gravity feature extraction model based on a spatiotemporal fusion convolutional architecture to generate spatiotemporal features of the center of gravity. The center of gravity feature fusion module is used to generate global center of gravity prediction features by using the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle and the trunk stability index through a feature fusion model based on an LSTM network and attention mechanism architecture. The center of gravity position prediction module is used to generate the predicted center of gravity position by passing the global center of gravity prediction features through a fully connected layer based on dynamic centroid guidance. An adjustment module is used to adjust the movements of the VR teaching and ski simulator based on the predicted center of gravity position.

2. The VR-based intelligent teaching system for skiing according to claim 1, characterized in that, The centroid feature fusion module includes: The LSTM temporal modeling unit is used to model the spatiotemporal features of the centroid through the LSTM network in the time dimension to generate a comprehensive hidden state. The data augmentation unit is used to perform weighted calculations on the spatiotemporal features of the center of gravity, the foot COP, the waist tilt angle, and the trunk stability index to generate enhanced attention input data. An attention mechanism computation unit is used to process the enhanced attention input data through an attention mechanism to generate attention fusion weights; The fusion unit is used to multiply the integrated hidden state and the attention fusion weights to generate the global centroid prediction features.

3. The VR-based intelligent teaching system for skiing according to claim 2, characterized in that, The enhanced attention input data includes enhanced key vectors and enhanced query vectors, and the data enhancement unit includes: An initial vector generation subunit is used to map the foot COP, waist tilt angle and trunk stability index through multiple first training weight matrices to generate an initial key vector and an initial query vector. An enhanced key vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial key vector using a second training weight matrix to generate an enhanced key vector. An enhanced query vector generation subunit is used to compress the centroid spatiotemporal features through a pooling layer, and then perform a weighted calculation with the initial query vector using a third training weight matrix to generate an enhanced query vector.

4. The VR-based intelligent teaching system for skiing according to claim 3, characterized in that, The enhancement key vector includes a foot enhancement key vector and a waist enhancement key vector; the attention fusion weight includes a foot attention fusion weight and a waist attention fusion weight; and the attention mechanism calculation unit includes: The foot attention calculation subunit is used to generate foot attention fusion weights by passing the enhanced query vector and the foot enhanced key vector through a first attention mechanism. The waist attention calculation subunit is used to generate waist attention fusion weights by passing the enhanced query vector and the waist enhanced key vector through a second attention mechanism. The fusion subunit is used to perform a first weighted summation of the foot attention fusion weight, foot value vector, foot value vector, waist attention fusion weight, and waist value vector, and then perform a second weighted summation with the trunk stability index to generate the global center of gravity prediction feature.

5. The VR-based intelligent teaching system for skiing according to claim 1, characterized in that, The center of gravity position prediction module includes: A mapping unit is used to pass the global centroid prediction features through a fully connected layer to generate initial mapping features; The dynamic center of mass calculation unit is used to calculate the dynamic center of mass distance based on the pressure data, foot speed collected by the foot sensor, waist speed collected by the waist sensor, hand speed collected by the hand sensor, and the set weight. The dynamic weight calculation unit is used to map the concatenated vector of the global center of gravity prediction features, foot velocity variance, and waist velocity variance to generate dynamic weights. The weighted correction unit is used to perform weighted calculations on the initial mapping features and the dynamic centroid distance based on the dynamic weights to generate the predicted centroid position.

6. The VR-based intelligent teaching system for skiing according to claim 5, characterized in that, The dynamic centroid calculation unit includes: An initial centroid calculation subunit is used to generate an initial centroid distance based on the pressure data and the set weight using a mass-weighted average method. The center of mass offset velocity calculation subunit is used to generate the center of mass offset velocity based on the foot velocity, the waist velocity, the hand velocity, and the set weight using a velocity-weighted average method. The summation calculation subunit is used to calculate the dynamic centroid distance based on the initial centroid distance and the centroid offset velocity.

7. The VR-based intelligent teaching system for skiing according to claim 1, characterized in that, The centroid feature extraction module includes: The convolutional temporal feature extraction unit is used to extract temporal features from the foot COP, waist tilt angle and trunk stability index through a one-dimensional convolutional layer to generate foot temporal features, waist temporal features and trunk stability temporal features. The spatial interaction unit is used to take the maximum value of the time dimension of the foot temporal features, the waist temporal features and the trunk stability temporal features respectively, and then perform feature splicing and mapping to generate the center of gravity spatiotemporal features.

8. The VR-based intelligent teaching system for skiing according to claim 1, characterized in that, The physical conversion module includes: A foot COP generation unit is used to calculate the foot COP based on the pressure data and pressure horizontal coordinates; The waist tilt angle generation unit is used to generate the waist tilt angle based on the arctangent function of the attitude data in the horizontal direction and the attitude data in the vertical direction; The trunk stability index generation unit is used to calculate the trunk stability index based on the standard deviation of the center position of the foot COP, the average value of the center position of the foot COP, and the rate of change of the waist tilt angle.

9. The VR-based intelligent teaching system for skiing according to claim 1, characterized in that, Also includes: The loss function construction module is used to construct a collaborative loss term based on regularization loss, which includes spatiotemporal fusion convolution parameters, LSTM network parameters, and attention mechanism parameters. It also constructs a centroid prediction loss term based on the Huber function. Based on the collaborative loss term and the centroid prediction loss term, a comprehensive loss function is constructed for collaborative training and optimization of the centroid feature extraction module, the centroid feature fusion module, and the centroid position prediction module.

10. The VR-based intelligent teaching system for skiing according to any one of claims 1 to 9, characterized in that, The adjustment module includes: The VR teaching early warning unit is used to adjust the early warning direction of VR teaching based on the predicted center of gravity position. The ski simulator angle adjustment unit is used to adjust the ski simulator angle based on the predicted center of gravity position using the PD algorithm.

Images