Dance motion capture and real-time feedback control system

Abstract

The invention relates to the technical field of dance training, and discloses a dance motion capture and real-time feedback control system which comprises a motion capture module, a motion analysis module, a real-time feedback control module and a motion execution module. Capturing dancing actions and physiological parameters of the dancer in real time through an action capturing technology to form a dancing action data set; training the dance movement recognition model and the movement evaluation model by using a deep learning algorithm to realize accurate recognition and multi-dimensional evaluation of the dance movement; constructing a feedback control model in combination with a reinforcement learning algorithm, and dynamically generating and adjusting a feedback control instruction according to an evaluation result; and finally, real-time correction and guidance of the dance movement are realized through the movement execution module. According to the method, timely and objective feedback can be provided for the dancer, the dancer is helped to adjust and optimize dance actions, the training efficiency and effect are improved, and the training consistency is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of dance training, in particular to a dance motion capture and real-time feedback control system. Background Art

[0002] In the field of dance teaching and performance, dance training methods mainly rely on the teacher's experience and intuitive judgment to guide and correct students' dance movements. This method has problems such as strong subjectivity, untimely feedback, and difficulty in quantitative evaluation, which leads to low training efficiency and difficulty in ensuring the objectivity and consistency of training effects. With the advancement of science and technology, especially the rapid development of computer vision, artificial intelligence and sensor technology, new technical means have been provided for dance training. As an important technology, motion capture technology can capture dancers' dance movements in real time and accurately, and convert them into digital data, which makes it possible to objectively analyze and evaluate dance movements. Existing motion capture technology is mainly used in animation production, game development, sports training and other fields. There are relatively few professional applications for dance training, and there is a lack of systematic solutions combined with real-time feedback control.

[0003] In dance training, dancers not only need to understand the correctness of their own movements, but also need timely feedback and guidance to adjust and optimize their dance movements. Although there are some dance training systems based on motion capture on the market, most of these systems can only provide simple movement recording and playback functions, lacking in-depth movement analysis and real-time feedback control mechanisms. Dancers often cannot intuitively understand the accuracy, completion, skill and fluency of their own movements, nor can they make timely adjustments and optimizations based on feedback results. Summary of the invention

[0004] The purpose of the present invention is to provide a dance motion capture and real-time feedback control system to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: a dance motion capture and real-time feedback control system, the system comprising:

[0006] The motion capture module is used to capture the dancer's dance movements in real time and collect the dancer's physiological parameters as reference data for motion analysis; the captured dance movement data is integrated and preprocessed with the reference data to form a dance movement data set;

[0007] The motion analysis module is used to build a dance motion recognition model and a motion evaluation model. The dance motion recognition model is used to determine the type of dance motion, and the motion evaluation model is used to evaluate the accuracy, completion, skill and fluency of the motion. The recognition model and the motion evaluation model are trained using a deep learning algorithm based on the dance motion data set.

[0008] The real-time feedback control module is used to generate real-time feedback control instructions using the outputs of the dance movement recognition model and the movement evaluation model; specifically, the feedback control model is constructed using a reinforcement learning algorithm, and the control results are quantitatively evaluated according to the accuracy, completion, skill, physiological parameters and movement fluency of the dance movements by defining an evaluation function, and the generated control instructions are dynamically adjusted based on the evaluation results;

[0009] The action execution module is used to receive feedback control instructions and drive the corresponding equipment or system to perform adjustment or guidance actions to achieve real-time correction of dance movements.

[0010] Preferably, the evaluation function is defined as follows: G(s,a)=k1*P(s,a)+k2*C(s,a)+k3*T(s,a)+k4*U(s,a)+k5*F(s,a), wherein s represents the current state space of the dancer, including position, velocity, acceleration and physiological parameters; a represents the control action taken; k1, k2, k3, k4, k5 are weight coefficients of recognition accuracy, completion, skill, physiological parameters and movement fluency, respectively, and k1+k2+k3+k4+k5=1; P(s,a), C(s,a), T(s,a), U(s,a), F(s,a) respectively represent the recognition accuracy, completion, skill, physiological parameter evaluation value and movement fluency evaluation value after taking action a in state s.

[0011] Preferably, the real-time feedback control module uses a deep Q network DQN algorithm in a reinforcement learning algorithm to train a feedback control model.

[0012] Preferably, the step of training the feedback control model includes:

[0013] Step 1: Set the hyperparameters of the reinforcement learning algorithm, including learning rate, discount factor, and exploration rate; initialize the policy network and value network, both of which use deep neural network structures. The policy network is used to output the action probability distribution under the current state, and the value network is used to estimate the value of the state or action-state pair;

[0014] Step 2: Define an experience replay pool to store samples of the dancer's interaction with the environment. Each sample contains the current state, the action taken, the immediate reward obtained, the next state, and a flag indicating whether it has terminated.

[0015] Step 3: The dancer’s dance movements are captured in real time through the motion capture layer and converted into state representations; actions are selected based on the current strategy network, executed in the virtual reality environment, and feedback from the environment is observed, including new state information and immediate rewards; interaction data is stored in the experience replay pool;

[0016] Step 4: Randomly draw a batch of samples from the experience replay pool; use the sample data to update the value network and optimize the network parameters by minimizing the error between the predicted value and the true value; use the sample data and the output of the value network to update the policy network and maximize the expected value of the cumulative reward through the policy gradient method;

[0017] Step 5: Repeat steps 3 to 4 until the preset training rounds are reached or the performance of the policy network is stable; during the training process, gradually reduce the exploration rate so that the policy network makes more use of known optimal actions rather than exploring new actions.

[0018] Preferably, the action execution module includes a virtual reality device control submodule and a dance auxiliary device control submodule; the virtual reality device control submodule is used to adjust the visual and auditory feedback in the virtual reality environment according to the control instructions, so as to provide the dancer with an immersive dance experience; the dance auxiliary device control submodule is used to adjust the movement trajectory, strength and speed of the dance auxiliary device according to the control instructions, so as to provide the dancer with physical real-time feedback.

[0019] Preferably, a convolutional neural network (CNN) algorithm is used to construct a dance movement recognition model, and features in dance movements are extracted through multi-layer convolution and pooling operations, specifically: a CNN model is defined, the number of convolution layers, the size of each convolution kernel, the pooling method, the type of activation function, the structure of the fully connected layer, and the classifier of the output layer are determined; wherein the convolution layer is used to extract local features of dance movements, the pooling layer is used to reduce the dimension of the feature map and retain important information, the fully connected layer is used to map high-dimensional features to the classification space, and the output layer uses a softmax classifier to give the probability distribution of dance movement types.

[0020] Preferably, the step of training the dance movement recognition model includes:

[0021] The preprocessed dance action dataset is input into the CNN model, and the loss between the model output and the true label is calculated through the forward propagation algorithm. The loss function adopts the cross entropy loss function; its formula is:

[0022]

[0023] Where N is the number of samples, y i is the true label of the i-th sample, p i is the predicted probability of the i-th sample;

[0024] The gradient of the loss function with respect to the model parameters is calculated through the back-propagation algorithm, and the model parameters are updated using the gradient descent algorithm;

[0025] The recognition accuracy of the trained CNN model is evaluated on an independent validation set, and the number of layers, convolution kernel size, and learning rate of the CNN model are adjusted according to the evaluation results.

[0026] Preferably, a recurrent neural network (RNN) algorithm is used to construct the action evaluation model.

[0027] Preferably, the step of training the action evaluation model comprises:

[0028] Arrange the dancer's dance movement records in chronological order to form dance movement time series data;

[0029] Construct an RNN model, determine the number of hidden layers, the number of hidden units in each layer, the type of activation function, the learning rate, and the optimizer; wherein the RNN model consists of recurrent units, each of which can receive the input at the current moment and the hidden state at the previous moment, and output the hidden state at the current moment, thereby realizing the memory and transmission of time series data; the hidden state update formula of the RNN model is:

[0030] h t =σ(W hh h t-1 +W xh x t +b h )

[0031] Among them, h t represents the hidden state at time t, x t represents the input at time t, W hh and W xh Represents the weight matrices from hidden layer to hidden layer and from input layer to hidden layer, b h represents the bias term of the hidden layer, σ represents the activation function;

[0032] Input the serialized dance movement data into the RNN model to learn the changing patterns of dance movements over time;

[0033] According to the error between the action evaluation result predicted by the RNN model and the actual action, the parameters and structure of the model are adjusted.

[0034] Preferably, the method of preprocessing the captured dance movement data and reference data includes:

[0035] Perform one-hot encoding conversion on discrete features;

[0036] The Z-score standardization method is used to standardize continuous features. Specifically, each numerical data is subtracted from its mean and divided by its standard deviation so that the processed data conforms to the standard normal distribution. The standardization formula is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] This system uses the motion capture module to capture the dancer's dance movements in real time, and combines physiological parameters for comprehensive analysis, achieving accurate capture and comprehensive evaluation of dance movements. Compared with the traditional method that relies on teacher experience and intuitive judgment, this system can significantly reduce the influence of subjectivity and provide more objective and accurate dance movement evaluation results, thereby greatly improving the efficiency and effectiveness of dance training. The real-time feedback control module can quickly generate feedback control instructions based on the output of the dance movement recognition model and the movement evaluation model, and realize instant movement correction and guidance through the movement execution module. This instant feedback mechanism enables dancers to immediately understand the shortcomings of their own movements, and adjust and optimize accordingly, effectively improving the accuracy and skill of dance movements.

[0039] The motion analysis module uses deep learning algorithms to deeply analyze and evaluate dance movements. It can not only identify the types of dance movements, but also comprehensively evaluate the accuracy, completion, skill and fluency of the movements. This in-depth motion analysis provides dancers with a more detailed motion evaluation report, which helps them to have a more comprehensive understanding of their own dance level and improve their dance skills in a targeted manner.

[0040] This system captures and evaluates dance movements in a digital way, avoiding the problem of inconsistency in training caused by differences in teachers' personal experience and subjective judgment in traditional methods. At the same time, the objective evaluation criteria provided by the system also ensure the fairness and impartiality of dance training, which helps to improve the overall quality and level of dance teaching. The professional application of this system is not limited to the field of dance teaching, but can also be extended to dance performances, dance choreography, dance research and other fields. By capturing and analyzing dance movements in real time, the system can provide dance creators with inspiration and movement materials, and provide dance researchers with rich data support, further promoting the development and innovation of dance art. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a working principle diagram of the dance motion capture and real-time feedback control system of the present invention;

[0042] Figure 2 Flowchart for building feedback control model;

[0043] Figure 3 Flowchart for building the action evaluation model. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] See also Figure 1-3 The present invention provides a technical solution: a dance motion capture and real-time feedback control system, the system comprising:

[0046] Motion capture module: This module uses high-precision motion capture equipment, such as optical motion capture systems, inertial sensors or depth cameras, to capture dancers' dance movements in real time. At the same time, physiological parameter monitoring equipment, such as heart rate monitors, muscle activity sensors, etc., are integrated to collect dancers' physiological parameters as auxiliary data for motion analysis. The captured dance movement data and physiological parameter data are integrated and preprocessed to form a unified dance movement data set. The preprocessing method for the captured dance movement data includes: performing one-hot encoding conversion on discrete features. The Z-score standardization method is used to standardize continuous features. Specifically, each numerical data is subtracted from its mean and divided by its standard deviation, so that the processed data conforms to the standard normal distribution. The formula for standardization is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

[0047] Action analysis module: In this module, we first build a dance action recognition model and an action evaluation model. The dance action recognition model is based on a deep learning algorithm and trains the dance action dataset to accurately identify different types of dance actions. The action evaluation model uses deep learning technology, combined with dance action datasets and expert annotation information, to evaluate the accuracy, completion, skill, and fluency of the action. The evaluation model provides support for the refined evaluation of dance actions through multi-dimensional analysis.

[0048] Real-time feedback control module: This module uses a reinforcement learning algorithm to build a feedback control model. By defining an evaluation function, the accuracy, completion, skill, physiological parameters and fluency of dance movements are comprehensively considered to quantitatively evaluate the control results. The feedback control model dynamically adjusts the generated control instructions based on the evaluation results to ensure the timeliness and effectiveness of the feedback. The control instructions include movement adjustment suggestions, training intensity adjustment, skill improvement guidance, etc., aiming to help dancers correct their movements immediately and improve training results.

[0049] Action execution module: This module receives the control instructions generated by the real-time feedback control module and drives the corresponding device or system to perform adjustment or guidance actions. For example, through virtual reality (VR) or augmented reality (AR) technology, action adjustment suggestions are displayed in real time in the dancer's training environment; or through smart wearable devices, such as vibration feedback belts, the dancer is directly given physical action correction guidance. The implementation methods of the action execution module are flexible and diverse, aiming to achieve real-time correction of dance movements and immediate improvement of training effects.

[0050] The present invention will be further described below in conjunction with Examples 1 to 3:

[0051] Embodiment 1:

[0052] The evaluation function G(s,a) is the core component in the real-time feedback control module, which is used to quantify the comprehensive effect of taking action a in a specific state s. The evaluation function is defined as: G(s,a) = k1*P(s,a) + k2*C(s,a) + k3*T(s,a) + k4*U(s,a) + k5*F(s,a), where s represents the current state space of the dancer, including position, velocity, acceleration and physiological parameters; a represents the control action taken; k1, k2, k3, k4, k5 are the weight coefficients of recognition accuracy, completion, skill, physiological parameters and movement fluency, respectively, and k1+k2+k3+k4+k5=1; P(s,a), C(s,a), T(s,a), U(s,a), F(s,a) represent the recognition accuracy, completion, skill, physiological parameter evaluation value and movement fluency evaluation value after taking action a in state s, respectively. These evaluation values ​​are calculated through the deep learning model in the motion analysis module, providing an accurate quantitative basis for feedback control.

[0053] The real-time feedback control module uses the deep Q network (DQN) algorithm in the reinforcement learning algorithm to train the feedback control model. The DQN algorithm combines the representation ability of deep learning and the decision-making ability of reinforcement learning, and is suitable for processing problems in high-dimensional state space and continuous action space.

[0054] The training steps of the feedback control model include:

[0055] Step 1: Set the hyperparameters of the reinforcement learning algorithm, including the learning rate (controlling the step size of the model parameter update), the discount factor (determining the weight of future rewards in the current value), and the exploration rate (controlling the ratio of the model exploring new actions to using known optimal actions during training). Initialize the policy network and the value network, both of which use deep neural network structures. The policy network is used to output the probability distribution of actions in the current state, that is, given a state s, the policy network predicts the probability of each possible action a; the value network is used to estimate the value of a state or action-state pair, that is, given a state s (or state-action pair s, a), the value network predicts the long-term cumulative reward of the state (or state-action pair).

[0056] Step 2: Define an experience replay pool to store samples of the dancer's interaction with the environment. Each sample contains the current state, the action taken, the immediate reward, the next state, and a flag indicating whether it is terminated. The use of the experience replay pool can improve sample utilization, break the temporal correlation between samples, and thus stabilize the training process.

[0057] Step 3 (interaction data collection): The dancer's dance movements are captured in real time through the motion capture layer and converted into state representations. Actions are selected based on the current policy network (usually obtained by sampling the probability distribution of actions) and executed in the virtual reality environment. Feedback from the environment is observed, including new state information and immediate rewards (rewards are calculated based on the evaluation function G(s,a)). The interaction data (state, action, reward, next state, termination flag) is stored in the experience replay pool.

[0058] Step 4 (Model Update): A batch of samples (called minibatch) are randomly drawn from the experience replay pool to update the value network and policy network. For the value network, the sample data (especially state, action and reward) is used to optimize the network parameters by minimizing the error (such as mean square error) between the predicted value and the true value. For the policy network, the sample data and the output of the value network (i.e., the value estimate of the action-state pair) are used to maximize the expected value of the cumulative reward through the policy gradient method.

[0059] Step 5 (iterative training): Repeat steps 3 to 4 until the preset training rounds are reached or the performance of the policy network is stable (such as the cumulative reward no longer increases significantly). During the training process, gradually reduce the exploration rate so that the policy network can use more known optimal actions rather than exploring new actions to improve training efficiency and model performance.

[0060] The action execution module includes a virtual reality device control submodule and a dance auxiliary device control submodule; the virtual reality device control submodule is responsible for adjusting the visual and auditory feedback in the virtual reality environment according to the control instructions generated by the real-time feedback control module. This submodule is implemented in the following way:

[0061] Visual feedback adjustment: When a dancer wears a virtual reality helmet or glasses, the virtual reality device control submodule can update the image on the helmet or glasses display in real time to reflect the dancer's position, movement, and changes in the surrounding environment in the virtual environment. For example, if the control instruction suggests that the dancer adjust the angle or speed of a dance move, the virtual reality environment will adjust the dancer's virtual image accordingly to make its movement more in line with the instruction requirements. At the same time, the submodule can also change the background, lighting, and special effects of the virtual environment to provide dancers with a richer and more immersive dance experience.

[0062] Auditory feedback adjustment: In addition to visual feedback, the submodule can also adjust the sound effects in the VR environment, such as music rhythm, sound effects, and narration. These sound effects can be adjusted in real time according to the dancer's movements and control instructions to provide more precise auditory guidance. For example, when the dancer needs to speed up or slow down the rhythm of the movement, the background music and rhythm cues will speed up or slow down accordingly, helping the dancer to better master the dance rhythm.

[0063] The dance auxiliary equipment control submodule is responsible for adjusting the movement trajectory, strength and speed of the dance auxiliary equipment according to the control instructions, providing real-time physical feedback to the dancer. This submodule is implemented in the following ways:

[0064] Motion trajectory adjustment: Dance assistive devices can include smart wearable devices (such as vibration feedback belts, smart bracelets, etc.) or robotic arms. When the control instruction requires the dancer to adjust the motion trajectory of a dance move, the submodule will control these devices to move along the specified trajectory to provide physical guidance for the dancer. For example, if the dancer needs to practice a complex dance step, the submodule can control the robotic arm to move along the correct trajectory to help the dancer feel the correct movement path.

[0065] Strength and speed adjustment: In addition to the motion trajectory, the submodule is also able to adjust the strength and speed of the dance assistive device. This can be achieved by changing the vibration frequency, intensity, or thrust of the device's robotic arm. For example, when the control instruction requires the dancer to increase the strength or speed of a certain movement, the submodule can control the vibration feedback belt to remind the dancer with stronger vibration or faster frequency, or control the robotic arm to assist the dancer to complete the movement with greater thrust.

[0066] Through the collaborative work of the virtual reality device control submodule and the dance auxiliary device control submodule, this dance motion capture and real-time feedback control system can provide dancers with all-round, multi-dimensional real-time feedback and correction suggestions. This feedback is not limited to the visual and auditory levels, but also includes direct guidance at the physical level, which greatly improves the efficiency and effectiveness of dance training. Dancers can practice freely in an immersive virtual environment, while getting accurate physical feedback and correction suggestions, and constantly optimizing their dance movements and techniques.

[0067] Embodiment 2:

[0068] The following will describe in detail the specific implementation method of using the convolutional neural network (CNN) algorithm to build a dance movement recognition model, as well as the steps for training the model, with specific examples.

[0069] ①The construction of dance movement recognition model includes:

[0070] Define the CNN model structure:

[0071] Convolutional layer: The convolutional layer is the core of the CNN model and is used to extract local features of dance movements. The number of convolutional layers is usually determined based on the complexity of the data set and the requirements of the model. The size of each convolution kernel determines the level of refinement of feature extraction. Smaller convolution kernels can capture more detailed features, while larger convolution kernels can capture more global features. In this embodiment, multiple convolutional layers can be selected, and each layer uses convolution kernels of different sizes to extract multi-scale features.

[0072] Pooling layer: The pooling layer follows the convolution layer and is used to reduce the dimension of the feature map and retain important information. Common pooling methods include maximum pooling and average pooling. In this embodiment, maximum pooling can be selected because it can retain the maximum value of each local area, thereby better retaining key features.

[0073] Activation function: The activation function introduces nonlinearity to the model, allowing the model to learn complex mapping relationships. Common activation functions include ReLU, Sigmoid, and Tanh. In this embodiment, the ReLU activation function can be selected because it has the advantages of simple calculation and gradient is not easy to disappear.

[0074] Fully connected layer: A fully connected layer is used to map high-dimensional features to a classification space. When determining the structure of a fully connected layer, the dimensions of the input features and the number of output categories need to be considered. Typically, a fully connected layer contains one or more hidden layers and an output layer.

[0075] Output layer: The output layer uses a softmax classifier to give the probability distribution of dance action types. The softmax function can convert the original scores of the output layer into probabilities so that the sum of the probabilities of all categories is 1.

[0076] ②The steps of training the dance movement recognition model include:

[0077] Data preprocessing: First, the dance movement dataset is preprocessed, including data cleaning, normalization, data enhancement and other operations to improve the generalization ability and training effect of the model.

[0078] Forward propagation calculation loss: The preprocessed dance action dataset is input into the CNN model, and the loss between the model output and the true label is calculated through the forward propagation algorithm. The loss function uses the cross entropy loss function, and its formula is:

[0079]

[0080] Where N is the number of samples, y i is the true label of the i-th sample, p i is the predicted probability of the ith sample.

[0081] Back propagation parameter update: The gradient of the loss function with respect to the model parameters is calculated through the back propagation algorithm, and the gradient descent algorithm is used to update the model parameters. When updating the parameters, it is necessary to set an appropriate learning rate to control the step size of the parameter update.

[0082] Model evaluation and adjustment: Evaluate the recognition accuracy of the trained CNN model on an independent validation set. Based on the evaluation results, you can adjust the CNN model's hyperparameters such as the number of layers, convolution kernel size, and learning rate to further improve the model's performance.

[0083] Iterative training: Repeat the above steps until the performance of the model on the validation set reaches a stable state or no longer improves significantly. During the training process, strategies such as early stopping and learning rate decay can be used to prevent overfitting and accelerate training.

[0084] Embodiment 3:

[0085] This embodiment is used to describe in detail the specific implementation method of using a recurrent neural network (RNN) algorithm to build an action evaluation model, which specifically includes:

[0086] The dancer's dance movement records are arranged in chronological order to form dance movement time series data. This can be achieved through motion capture equipment (such as motion capture suits, depth cameras, etc.), which converts the dancer's movements into a series of time point data, each of which contains the dancer's posture information (such as joint angles, positions, etc.).

[0087] Build the RNN model and determine the number of hidden layers and the number of hidden units in each layer. The number of hidden layers and the number of hidden units determine the complexity and learning ability of the model. Generally speaking, more complex tasks may require more hidden layers and hidden units.

[0088] Select an activation function. The activation function introduces nonlinearity to the model, allowing the model to learn complex mapping relationships. In RNN, commonly used activation functions include Sigmoid, Tanh, and ReLU. Select a suitable activation function based on the characteristics of the task and the performance requirements of the model.

[0089] Set the learning rate and optimizer. The learning rate controls the step size of the model parameter update, while the optimizer determines the way the parameters are updated. Select the Adam function as the optimizer.

[0090] The RNN model consists of recurrent units, each of which receives the input x at the current moment. t and the hidden state h at the previous moment t-1 , output the hidden state h at the current moment t The hidden state update formula is:

[0091] h t =σ(W hh h t-1 +W xh x t +b h )

[0092] Among them, h t represents the hidden state at time t, x t represents the input at time t, W hh and W xh Represents the weight matrices from hidden layer to hidden layer and from input layer to hidden layer, b h represents the bias term of the hidden layer, and σ represents the activation function.

[0093] Input the serialized dance action data into the RNN model. During the training process, the data will be input into the model in time steps, and the model will learn the change pattern of dance action over time. According to the error between the action evaluation result predicted by the RNN model and the actual action, the loss function is calculated. Common loss functions include mean square error (MSE), cross entropy loss, etc. Choose the appropriate loss function according to the characteristics of the task.

[0094] The gradient of the loss function with respect to the model parameters is calculated by the back propagation algorithm, and the model parameters are updated according to the gradient descent method. This process is repeated until the performance of the model on the training set reaches stability. The formula for updating the model parameters by the gradient descent method is:

[0095]

[0096] Among them, θ represents the model parameters, including all weights and biases that the model needs to learn; ω represents the learning rate, which is used to control the step size of parameter update; represents the gradient of the loss function J(θ) with respect to the parameter θ, is a vector pointing to the direction where the loss function grows fastest; := represents the assignment operation, that is, updating the value of the parameter θ.

[0097] According to the performance of the model on the validation set, the parameters and structure of the model are adjusted, such as increasing or decreasing the number of hidden layers, adjusting the number of hidden units, changing the learning rate, etc., to further improve the performance of the model; and the performance of the trained RNN model is evaluated on an independent test set, such as accuracy, recall rate, F1 score, etc. The trained RNN model is applied to the actual dance motion capture and real-time feedback control system to evaluate the dancer's movements in real time, and provide feedback and guidance based on the evaluation results.

[0098] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0099] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. Dance motion capture and real-time feedback control system, characterized in that: The system comprises: The motion capture module is used to capture the dancer's dance movements in real time and collect the dancer's physiological parameters as reference data for motion analysis; the captured dance movement data is integrated and preprocessed with the reference data to form a dance movement data set; The motion analysis module is used to build a dance motion recognition model and a motion evaluation model. The dance motion recognition model is used to determine the type of dance motion, and the motion evaluation model is used to evaluate the accuracy, completion, skill and fluency of the motion. The recognition model and the motion evaluation model are trained using a deep learning algorithm based on the dance motion data set. The real-time feedback control module is used to generate real-time feedback control instructions using the outputs of the dance movement recognition model and the movement evaluation model; specifically, the feedback control model is constructed using a reinforcement learning algorithm, and the control results are quantitatively evaluated according to the accuracy, completion, skill, physiological parameters and movement fluency of the dance movements by defining an evaluation function, and the generated control instructions are dynamically adjusted based on the evaluation results; The action execution module is used to receive feedback control instructions and drive the corresponding equipment or system to perform adjustment or guidance actions to achieve real-time correction of dance movements.

2. The dance motion capture and real-time feedback control system according to claim 1, characterized in that: The evaluation function is defined as follows: G(s,a)=k1*P(s,a)+k2*C(s,a)+k3*T(s,a)+k4*U(s,a)+k5*F(s,a), wherein s represents the current state space of the dancer, including position, velocity, acceleration and physiological parameters; a represents the control action taken; k1, k2, k3, k4 and k5 are weight coefficients of recognition accuracy, completion, skill, physiological parameters and movement fluency, respectively, and k1+k2+k3+k4+k5=1; P(s,a), C(s,a), T(s,a), U(s,a) and F(s,a) respectively represent the recognition accuracy, completion, skill, physiological parameter evaluation value and movement fluency evaluation value after taking action a in state s.

3. The dance motion capture and real-time feedback control system according to claim 2, characterized in that: The real-time feedback control module uses the deep Q network DQN algorithm in the reinforcement learning algorithm to train the feedback control model.

4. The dance motion capture and real-time feedback control system according to claim 3, characterized in that: The steps to train the feedback control model include: Step 1: Set the hyperparameters of the reinforcement learning algorithm, including learning rate, discount factor, and exploration rate; initialize the policy network and value network, both of which use deep neural network structures. The policy network is used to output the action probability distribution under the current state, and the value network is used to estimate the value of the state or action-state pair; Step 2: Define an experience replay pool to store samples of the dancer's interaction with the environment. Each sample contains the current state, the action taken, the immediate reward obtained, the next state, and a flag indicating whether it has terminated. Step 3: The dancer’s dance movements are captured in real time through the motion capture layer and converted into state representations; actions are selected based on the current strategy network, executed in the virtual reality environment, and feedback from the environment is observed, including new state information and immediate rewards; interaction data is stored in the experience replay pool; Step 4: Randomly draw a batch of samples from the experience replay pool; use the sample data to update the value network and optimize the network parameters by minimizing the error between the predicted value and the true value; use the sample data and the output of the value network to update the policy network and maximize the expected value of the cumulative reward through the policy gradient method; Step 5: Repeat steps 3 to 4 until the preset training rounds are reached or the performance of the policy network is stable; during the training process, gradually reduce the exploration rate so that the policy network makes more use of known optimal actions rather than exploring new actions.

5. The dance motion capture and real-time feedback control system according to claim 4, characterized in that: The action execution module includes a virtual reality device control submodule and a dance auxiliary device control submodule; the virtual reality device control submodule is used to adjust the visual and auditory feedback in the virtual reality environment according to the control instructions, so as to provide the dancer with an immersive dance experience; the dance auxiliary device control submodule is used to adjust the movement trajectory, strength and speed of the dance auxiliary device according to the control instructions, so as to provide the dancer with physical real-time feedback.

6. The dance motion capture and real-time feedback control system according to claim 1, characterized in that: A dance movement recognition model was constructed using the convolutional neural network (CNN) algorithm. Features in dance movements were extracted through multi-layer convolution and pooling operations. Specifically, the CNN model was defined to determine the number of convolutional layers, the size of the convolution kernel in each layer, the pooling method, the type of activation function, the structure of the fully connected layer, and the classifier of the output layer. The convolutional layer was used to extract local features of dance movements, the pooling layer was used to reduce the dimension of the feature map and retain important information, the fully connected layer was used to map high-dimensional features to the classification space, and the output layer used a softmax classifier to give the probability distribution of dance movement types.

7. The dance motion capture and real-time feedback control system according to claim 6, characterized in that: The steps to train a dance movement recognition model include: The preprocessed dance action dataset is input into the CNN model, and the loss between the model output and the true label is calculated through the forward propagation algorithm. The loss function adopts the cross entropy loss function; its formula is: Where N is the number of samples, y i is the true label of the i-th sample, p i is the predicted probability of the i-th sample; The gradient of the loss function with respect to the model parameters is calculated through the back-propagation algorithm, and the model parameters are updated using the gradient descent algorithm; The recognition accuracy of the trained CNN model is evaluated on an independent validation set, and the number of layers, convolution kernel size, and learning rate of the CNN model are adjusted according to the evaluation results.

8. The dance motion capture and real-time feedback control system according to claim 1, characterized in that: The recurrent neural network (RNN) algorithm is used to construct the action evaluation model.

9. The dance motion capture and real-time feedback control system according to claim 8, characterized in that: The steps to train the action evaluation model include: Arrange the dancer's dance movement records in chronological order to form dance movement time series data; Construct an RNN model, determine the number of hidden layers, the number of hidden units in each layer, the type of activation function, the learning rate, and the optimizer; wherein the RNN model consists of recurrent units, each of which can receive the input at the current moment and the hidden state at the previous moment, and output the hidden state at the current moment, thereby realizing the memory and transmission of time series data; the hidden state update formula of the RNN model is: h t =σ(W hh h t-1 +W xh x t +b h ) Among them, h t represents the hidden state at time t, x t represents the input at time t, W hh and W xh Represents the weight matrices from hidden layer to hidden layer and from input layer to hidden layer, b h represents the bias term of the hidden layer, σ represents the activation function; Input the serialized dance movement data into the RNN model to learn the changing patterns of dance movements over time; According to the error between the action evaluation result predicted by the RNN model and the actual action, the parameters and structure of the model are adjusted.

10. The dance motion capture and real-time feedback control system according to claim 1, characterized in that: The method of preprocessing the captured dance movement data and the reference data includes: Perform one-hot encoding conversion on discrete features; The Z-score standardization method is used to standardize continuous features. Specifically, each numerical data is subtracted from its mean and divided by its standard deviation so that the processed data conforms to the standard normal distribution. The standardization formula is: Z = (X-μ) / σ, where X is the original data, μ is the mean of the original data, σ is the standard deviation of the original data, and Z is the standardized data.

Images