Dance motion auxiliary generation method and system based on three-dimensional modeling

By constructing a three-dimensional human skeleton model and deep learning technology, combining kinematic constraints and temporal convolutional networks with graph convolutional neural networks, high-precision, smooth and beautiful dance movements are generated, solving the problems of unnatural movement generation and lack of aesthetic features in existing methods.

CN120765860AActive Publication Date: 2025-10-10SICHUAN TECH & BUSINESS UNIV

Patent Information

Application Number
CN202511281253.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-09-09
Publication Date
2025-10-10
Estimated Expiration
2045-09-09

AI Technical Summary

Technical Problem

Existing dance movement generation methods have the disadvantages of low accuracy, unnatural movement generation, and neglect of kinematic constraints and aesthetic characteristics, resulting in the generated movements lacking in details, physiology and aesthetics.

Method used

By constructing a three-dimensional human skeleton model, combining kinematic constraints and deep learning technology, and using the fusion of temporal convolutional networks and graph convolutional neural networks to generate dance movements, the smoothness and aesthetics of the candidate movements are comprehensively evaluated.

Benefits of technology

It achieves high-precision, smooth and beautiful dance movement generation, ensuring that the movements conform to the laws of human movement and artistic beauty.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120765860A_ABST
    Figure CN120765860A_ABST
Patent Text Reader

Abstract

The invention discloses a dance movement auxiliary generation method and system based on three-dimensional modeling. The method comprises the following steps: constructing a three-dimensional human skeleton model; generating a joint action set of each joint node in the dance action; constructing a dance movement auxiliary model based on fusion of a time sequence convolutional network and a graph convolutional neural network, obtaining three-dimensional joint movement trajectory data in continuous frames, and obtaining relevance among dance movements by analyzing time sequence features and collaborative movement rules in the dance movements; acquiring joint actions associated with the current dance action in the joint action set, and generating a candidate action set; and calculating to obtain a connection fluency score and a posture aesthetics score with the current dance movement, and weighting to obtain the candidate movement with the highest total score as the next dance movement. The method has the advantages that through precise skeleton modeling and kinematics constraint, the deep learning technology is combined, smooth and attractive dance actions are efficiently generated, and the method is suitable for dance creation and virtual performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to movement generation, and in particular to a dance movement auxiliary generation method and system based on three-dimensional modeling. Background Art

[0002] With the rapid development of artificial intelligence (AI), particularly the application of deep learning algorithms to image and video processing, dance movement generation technology has made significant progress. These technologies can automatically generate new dance moves or assist in their creation by learning and analyzing large amounts of dance movement data. For dancers and amateurs, these assisted generation methods can provide personalized training plans and real-time movement feedback, helping them improve their skills.

[0003] The dance movement auxiliary generation methods currently available on the market usually have the disadvantages of low accuracy and unnatural movement generation. Many methods rely on 2D image processing and cannot effectively capture the complex three-dimensional human skeleton and spatial movement, resulting in a lack of accuracy in the details of the generated movements, and are prone to inappropriate joint positions or posture distortion. In addition, existing methods often ignore kinematic constraints and physical limitations, resulting in physiologically unreasonable generated movements and prone to postures or movements that are beyond the capabilities of the human body. Although some methods combine deep learning technology, most of them only focus on the processing of temporal information and ignore the spatial correlation between joints, resulting in a lack of coherence and fluency in the generated dance movements. More importantly, existing methods usually do not comprehensively consider the aesthetic characteristics of the movement, such as the stretch, symmetry and balance of the posture, and the generated movements are relatively lacking in aesthetics. Summary of the Invention

[0004] In order to improve the existing methods and systems, a dance movement auxiliary generation method and system based on three-dimensional modeling is provided. This method can efficiently generate smooth and beautiful dance movements through precise skeleton modeling and kinematic constraints, combined with deep learning technology, and is suitable for dance creation and virtual performances.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] The dance movement auxiliary generation method based on three-dimensional modeling includes:

[0007] Obtain the initial posture data of the target dance and construct a 3D human skeleton model;

[0008] Based on the 3D human skeleton model, for each joint node, based on its kinematic constraints, a set of joint actions allowed for the joint in dance movements is generated;

[0009] A dance movement assistance model based on the fusion of a temporal convolutional network and a graph convolutional neural network was constructed. A large amount of labeled dance movement video data was input for dance movement assistance model training. The three-dimensional joint motion trajectory data in consecutive frames was obtained. Based on the dance movement assistance model, the correlation between the various dance movements was obtained by analyzing the time series characteristics of joint motion and the coordinated movement laws in the dance movements.

[0010] Based on the trained dance movement auxiliary model, the 3D human skeleton data of the current dance movement is input into the dance movement auxiliary model. Combined with the motion parameters allowed for each joint in the joint movement set, the joint movements in the joint movement set that are relevant to the current dance movement are obtained, and a candidate movement set is generated.

[0011] Based on the joint movements in the candidate movement set, the connection smoothness score and posture beauty score between the current dance movement are calculated, and the candidate movement with the highest total score is obtained by weighting as the next dance movement output.

[0012] Preferably, the step of obtaining the initial posture data of the target dance and constructing a three-dimensional human skeleton model specifically includes:

[0013] Using existing posture estimation tools, extract key human body data in each frame, including shoulders, elbows, knees, ankles, and wrists, and obtain their 3D coordinate data;

[0014] Preprocess the 3D coordinate data to generate dance posture data, and map it onto the 3D human skeleton model;

[0015] Correct the posture of the skeleton in the 3D human skeleton model to ensure that the angle and position of each joint are consistent with the actual dance movements.

[0016] Preferably, the method of generating a set of motion parameters allowed for each joint in a dance movement based on the three-dimensional human skeleton model and its kinematic constraints specifically includes:

[0017] Based on the physiological structure and physical limitations of each joint, motion constraints are constructed to obtain degrees of freedom. Constraints include the maximum angle, minimum angle, and rotation direction of the joint, and degrees of freedom include rotational and translational degrees of freedom.

[0018] A joint motion model is constructed based on constraints and degrees of freedom to obtain all possible actions of each joint and generate a set of joint actions, including discrete values ​​of joint angles, speed and acceleration limits, and timing constraints.

[0019] Preferably, the construction of a dance movement auxiliary model based on the fusion of a temporal convolutional network and a graph convolutional neural network, inputting a large amount of labeled dance movement video data to perform dance movement auxiliary model training, and obtaining three-dimensional joint motion trajectory data in continuous frames specifically includes:

[0020] The dance action video data is captured continuously through a temporal convolutional network, and the three-dimensional position data of each joint in each frame is obtained. The time series characteristics of the joint movement are obtained to classify and identify the dance action categories.

[0021] The adjacency relationship between joints is obtained through graph convolutional neural networks, and joints are regarded as nodes in the graph, and bone association relationships are regarded as edges of the graph;

[0022] The time series features obtained by the temporal convolutional network are fused with the spatial features obtained by the graph convolutional network to construct a dance movement auxiliary model.

[0023] Preferably, the dance movement auxiliary model based on the completed training inputs the three-dimensional human skeleton data of the current dance movement into the dance movement auxiliary model, combines the motion parameters allowed by each joint in the joint movement set, obtains the joint movements in the joint movement set that are associated with the current dance movement, and generates a candidate movement set, specifically including:

[0024] Inputting the 3D human skeleton data of the current dance movement into the trained dance movement auxiliary model to obtain the joint movements associated with the current dance movement;

[0025] Based on the motion parameters allowed for all joints in the joint action set, the joint action associated with the current dance action is matched with the joint actions in the joint action set to obtain a matching probability distribution;

[0026] Based on the joint actions in the joint action set that are higher than a preset threshold in the matching probability distribution, a candidate action set is generated.

[0027] Preferably, the step of calculating the connection smoothness score and posture beauty score between the joint movements in the candidate movement set and the current dance movement, and obtaining the candidate movement with the highest total score as the next dance movement output by weighted means specifically includes:

[0028] Based on the joint movements in the candidate movement set, the difference between the joint position of the current dance movement and the position of the corresponding joint in the candidate movement is obtained, and the position difference degree is calculated;

[0029] By calculating the speed of the joints in the current dance movement and the candidate movement, the speed connection fluency is calculated;

[0030] Score the motion connection fluency based on the position difference degree and the speed connection fluency;

[0031] Score the posture beauty degree based on the symmetry, balance, stability of the motion and the stretch, tension of the posture;

[0032] Based on the motion connection fluency score and the posture beauty degree score, the weighted total score of each candidate motion is obtained through comprehensive weighting, and the candidate motion with the highest score is selected as the next dance motion.

[0033] Further, a dance motion auxiliary generation system based on three-dimensional modeling is proposed, comprising:

[0034] A three-dimensional human skeleton module: the three-dimensional human skeleton module is used to extract the initial posture data of the target dance and construct a three-dimensional human skeleton model, so as to ensure that the angle and position of each joint meet the actual dance motion;

[0035] A motion parameter module: the motion parameter module generates a set of allowed motion parameters for each joint based on kinematic constraints, including joint angle, speed and acceleration limit;

[0036] A dance motion auxiliary model module: the dance motion auxiliary model module is used to fuse a time series convolution network and a graph convolution neural network, analyze a large amount of labeled dance motion video data, extract time sequence features, and obtain the correlation between dance motions;

[0037] A candidate motion set module: the candidate motion set module is used to input the three-dimensional human skeleton data of the current dance motion, combine the joint motion set to generate a candidate motion set associated with the current motion;

[0038] A scoring module: the scoring module is used to calculate the connection fluency and posture beauty degree score of the candidate motion and the current dance motion, and obtain the candidate motion with the highest total score as the next dance motion output through weighting;

[0039] A processor: the processor is used to process the calculation process of each formula and the construction calculation process of each model.

[0040] Compared with the prior art, the advantages of the present application are:

[0041] By constructing an accurate three-dimensional human skeleton model, this method can efficiently capture the details of dance movements, ensuring that the posture of each joint is consistent with the actual movement, and providing a high degree of accuracy. Secondly, by combining kinematic constraints and physical restrictions to generate a set of allowable movements for each joint, it can ensure that the generated dance movements are physiologically consistent with the laws of human movement and avoid unnatural or inappropriate movements. By fusing temporal convolutional networks with graph convolutional neural networks, the method can not only extract temporal and spatial features from a large number of annotated videos, but also effectively identify the correlation between dance movements, making the generated movements more smooth and coherent. Finally, by comprehensively scoring the smoothness of the connection and the beauty of the posture of the candidate movements, it ensures that the output dance movements are not only natural and smooth, but also have artistic beauty. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the method proposed in the present invention;

[0043] Figure 2 This is a schematic diagram of the three-dimensional human skeleton model proposed in the present invention;

[0044] Figure 3 This is a schematic diagram of the motion parameter set proposed by the present invention;

[0045] Figure 4 This is a schematic diagram of the association of dance movements proposed by the present invention;

[0046] Figure 5 Schematic diagram of the candidate action set proposed by the present invention;

[0047] Figure 6 This is a schematic diagram of the next dance movement output proposed by the present invention. DETAILED DESCRIPTION

[0048] The following description is intended to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are merely examples, and those skilled in the art may conceive of other obvious variations.

[0049] The dance movement auxiliary generation system based on 3D modeling includes:

[0050] 3D human skeleton module: The 3D human skeleton module is used to extract the initial posture data of the target dance and construct a 3D human skeleton model to ensure that the angle and position of each joint are consistent with the actual dance movements;

[0051] Motion parameter module: The motion parameter module generates a set of allowed motion parameters for each joint based on kinematic constraints, including joint angle, velocity, and acceleration limits;

[0052] Dance movement auxiliary model module: The dance movement auxiliary model module is used to integrate the time series convolutional network and the graph convolutional neural network to analyze a large amount of labeled dance movement video data, extract time series features, and obtain the correlation between dance movements;

[0053] Candidate action set module: The candidate action set module is used to input the three-dimensional human skeleton data of the current dance action, and generate a candidate action set associated with the current action in combination with the joint action set;

[0054] Scoring module: The scoring module is used to calculate the connection smoothness and posture beauty scores between the candidate movements and the current dance movement, and obtain the candidate movement with the highest total score through weighting as the next dance movement output;

[0055] Processor: The processor is used to process the calculation process of each formula and the construction calculation process of each model.

[0056] See Figure 1 As shown, the dance movement auxiliary generation method based on three-dimensional modeling includes:

[0057] Step 1: Obtain the initial posture data of the target dance and construct a 3D human skeleton model;

[0058] Step 2: Based on the 3D human skeleton model, for each joint node, based on its kinematic constraints, generate a set of joint movements allowed for the joint in the dance movement;

[0059] Step 3: Build a dance movement assistance model based on the fusion of a temporal convolutional network and a graph convolutional neural network. Input a large amount of labeled dance movement video data to train the dance movement assistance model, obtain the three-dimensional joint motion trajectory data in consecutive frames, and based on the dance movement assistance model, obtain the correlation between each dance movement by analyzing the time series characteristics of joint motion and the coordinated movement rules in the dance movement;

[0060] Step 4: Based on the trained dance movement auxiliary model, the 3D human skeleton data of the current dance movement is input into the dance movement auxiliary model. Combined with the allowed motion parameters of each joint in the joint movement set, the joint movements in the joint movement set that are related to the current dance movement are obtained, and a candidate movement set is generated.

[0061] Step 5: Based on the joint movements in the candidate movement set, calculate the connection smoothness score and posture beauty score between the current dance movement and the current dance movement, and obtain the candidate movement with the highest total score through weighted analysis as the next dance movement output.

[0062] See Figure 2 As shown in FIG, obtaining the initial posture data of the target dance and constructing a 3D human skeleton model specifically include:

[0063] Using existing posture estimation tools, extract key human body data in each frame, including shoulders, elbows, knees, ankles, and wrists, and obtain their 3D coordinate data;

[0064] Preprocess the 3D coordinate data to generate dance posture data, and map it onto the 3D human skeleton model;

[0065] Correct the posture of the skeleton in the 3D human skeleton model to ensure that the angle and position of each joint are consistent with the actual dance movements.

[0066] Specifically, existing pose estimation tools are used to extract human key points, including shoulders, elbows, knees, ankles, and wrists, from the target dance video or sequence frames. The pre-processed 3D coordinates are then mapped to the skeleton, i.e., a 3D human skeleton is constructed. The human skeleton is defined by the relative positions of the key points. The angles between each pair of joints are calculated based on the relative positions of the joints.

[0067] In order to ensure that the posture of the skeleton model conforms to the actual dance movements, it is necessary to correct the data of each frame and the joint angles in each frame to ensure that the angle changes between each joint of the skeleton and the adjacent joints are smooth and natural.

[0068] See Figure 3 As shown, based on the 3D human skeleton model, for each joint node, based on its kinematic constraints, a set of motion parameters allowed for the joint in dance movements is generated, specifically including:

[0069] Based on the physiological structure and physical limitations of each joint, motion constraints are constructed to obtain degrees of freedom. Constraints include the maximum angle, minimum angle, and rotation direction of the joint, and degrees of freedom include rotational and translational degrees of freedom.

[0070] A joint motion model is constructed based on constraints and degrees of freedom to obtain all possible actions of each joint and generate a set of joint actions, including discrete values ​​of joint angles, speed and acceleration limits, and timing constraints.

[0071] Specifically, the movement of each joint is subject to physical and physiological limitations. Different joints have different ranges of motion. For example, the maximum flexion angle of the elbow joint may be 150 degrees, and the minimum angle is 0 degrees. The rotation direction and axis of the same joint are also constrained. For example, the shoulder joint can rotate along multiple axes, but the knee joint can only rotate along a single axis. The specific rotation direction restrictions can be expressed using rotation matrices or quaternions.

[0072] The degrees of freedom of each joint define the number of directions in which the joint can move freely. The rotational degrees of freedom are the rotational degrees of freedom of the joint around a certain axis. For example, the elbow and knee joints each have 1 rotational degree of freedom, and the shoulder joint has 3 rotational degrees of freedom. The translational degrees of freedom are the displacement degrees of freedom of the joint, which describe the change in the position of the joint. For example, the hip joint usually has 3 translational degrees of freedom in three-dimensional space.

[0073] See Figure 4 As shown in the figure, a dance movement auxiliary model based on the fusion of temporal convolutional network and graph convolutional neural network is constructed. A large amount of labeled dance movement video data is input to train the dance movement auxiliary model, and the three-dimensional joint motion trajectory data in continuous frames is obtained. Specifically, it includes:

[0074] The dance action video data is captured continuously through a temporal convolutional network, and the three-dimensional position data of each joint in each frame is obtained. The time series characteristics of the joint movement are obtained to classify and identify the dance action categories.

[0075] The adjacency relationship between joints is obtained through graph convolutional neural networks, and joints are regarded as nodes in the graph, and bone association relationships are regarded as edges of the graph;

[0076] The time series features obtained by the temporal convolutional network are fused with the spatial features obtained by the graph convolutional network to construct a dance movement auxiliary model.

[0077] Specifically, the temporal convolutional network extracts temporal features through convolution operations. For the joint data in each frame of video, TCN extracts the time series features through the following formula:

[0078]

[0079] in, is the time series feature of joint j at time t, is the convolution kernel, is the bias term, K is the size of the convolution kernel, and the time series features of the joints at different time steps are obtained by sliding window calculation. is the original input data of joint j;

[0080] Through continuous temporal convolutional layers, the motion features of each joint in each frame can be obtained from the video data. The extracted time series features will be used for subsequent dance movement classification and analysis;

[0081] Each joint in the human skeleton is considered a node in the graph, and the connections between joints are considered edges in the graph. A graph is constructed, and the edge set represents the spatial connection between joints. The graph convolutional network performs convolution operations on the graph nodes through the adjacency matrix to extract the spatial features between joints, taking into account the spatial topology of the skeleton.

[0082] The outputs of the temporal convolutional network and the graph convolutional network are fused to build a more powerful dance movement auxiliary model, measure the similarity between different dance movements, and identify the collaborative movement relationship between different joints.

[0083] See Figure 5 As shown in the figure, based on the trained dance movement auxiliary model, the 3D human skeleton data of the current dance movement is input into the dance movement auxiliary model. Combined with the motion parameters allowed for each joint in the joint movement set, the joint movements in the joint movement set that are associated with the current dance movement are obtained, and a candidate movement set is generated, specifically including:

[0084] Inputting the 3D human skeleton data of the current dance movement into the trained dance movement auxiliary model to obtain the joint movements associated with the current dance movement;

[0085] Based on the motion parameters allowed for all joints in the joint action set, the joint action associated with the current dance action is matched with the joint actions in the joint action set to obtain a matching probability distribution;

[0086] Based on the joint actions in the joint action set that are higher than a preset threshold in the matching probability distribution, a candidate action set is generated.

[0087] Specifically, action matching is performed based on the action data of the current joint and all allowed motion parameters in the joint action set. The matching degree between the current joint action and each joint action in the action set is calculated to generate a matching probability distribution. By setting a preset threshold, joint actions with a matching degree higher than the threshold are screened out, thereby generating a candidate action set.

[0088] The candidate action set contains all joint actions with a matching degree above a threshold. These actions are selected based on the matching of the 3D skeleton data of the current dance action with the motion parameters in the joint action set, and the joint actions that are most likely to belong to the current dance action are selected.

[0089] See Figure 6 As shown, based on each joint action in the candidate action set, the connection smoothness score and posture beauty score between the current dance action are calculated, and the candidate action with the highest total score is obtained by weighting as the next dance action output. Specifically, it includes:

[0090] Based on the joint movements in the candidate movement set, the difference between the joint position of the current dance movement and the position of the corresponding joint in the candidate movement is obtained, and the position difference degree is calculated;

[0091] By calculating the speed of the joints in the current dance movement and the candidate movement, the speed connection fluency is calculated;

[0092] The smoothness of movement transitions is scored based on positional differences and speed transition fluency;

[0093] The posture aesthetics is scored based on movement symmetry, balance, stability, and the stretch and tension of the posture;

[0094] Based on the movement connection smoothness score and the posture beauty score, the weighted total score of each candidate movement is obtained through comprehensive weighting, and the candidate movement with the highest score is selected as the next dance movement.

[0095] Specifically, the position difference is used to measure the difference in joint position between the current dance movement and the candidate movement. It is calculated through Euclidean distance and the total position difference of the entire movement is obtained by weighted average of the differences of all joints. The speed connection smoothness reflects the smoothness of the connection in speed between the current dance movement and the candidate movement.

[0096] The symmetry of the movement is evaluated by calculating the position difference between the left and right joints, and the flexibility and tension are measured by the angle change between the joints. Based on this, the aesthetics of the posture is scored.

[0097] The final movement score is obtained by weighting the movement connection smoothness score and the posture beauty score. By sorting the weighted total score of all candidate movements, the candidate movement with the highest score is selected as the next dance movement.

[0098] In summary, the advantages of the present invention are: by constructing an accurate three-dimensional human skeleton model, the method can efficiently capture the details of dance movements, ensure that the posture of each joint is consistent with the actual movement, and provide a high degree of accuracy. Secondly, by combining kinematic constraints and physical limitations to generate a set of allowable movements for each joint, it can be ensured that the generated dance movements are physiologically consistent with the laws of human movement and avoid unnatural or inappropriate movements. By fusing the temporal convolutional network with the graph convolutional neural network, the method can not only extract temporal and spatial features from a large number of annotated videos, but also effectively identify the correlation between dance movements, making the generated movements more fluent and coherent. Finally, by comprehensively scoring the connection smoothness and posture beauty of the candidate movements, it is ensured that the output dance movements are not only natural and fluent, but also have artistic beauty.

[0099] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] The various embodiments in the specification are described in progressive manner, and the same or similar parts between the various embodiments can be mutually referred to, and each embodiment focuses on the difference from other embodiments.

[0101] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A dance movement auxiliary generation method based on three-dimensional modeling, characterized in that: include: Obtain the initial posture data of the target dance and construct a 3D human skeleton model; Based on the 3D human skeleton model, for each joint node, based on its kinematic constraints, a set of joint actions allowed for the joint in dance movements is generated; A dance movement assistance model based on the fusion of a temporal convolutional network and a graph convolutional neural network was constructed. A large amount of labeled dance movement video data was input for dance movement assistance model training. The three-dimensional joint motion trajectory data in consecutive frames was obtained. Based on the dance movement assistance model, the correlation between the various dance movements was obtained by analyzing the time series characteristics of joint motion and the coordinated movement laws in the dance movements. Based on the trained dance movement auxiliary model, the 3D human skeleton data of the current dance movement is input into the dance movement auxiliary model. Combined with the motion parameters allowed for each joint in the joint movement set, the joint movements in the joint movement set that are relevant to the current dance movement are obtained, and a candidate movement set is generated. Based on the joint movements in the candidate movement set, the connection smoothness score and posture beauty score between the current dance movement are calculated, and the candidate movement with the highest total score is obtained by weighting as the next dance movement output.

2. The dance movement auxiliary generation method based on three-dimensional modeling according to claim 1 is characterized in that: The step of obtaining the initial posture data of the target dance and constructing a three-dimensional human skeleton model specifically includes: Using existing posture estimation tools, extract key human body data in each frame, including shoulders, elbows, knees, ankles, and wrists, and obtain their 3D coordinate data; Preprocess the 3D coordinate data to generate dance posture data, and map it onto the 3D human skeleton model; Correct the posture of the skeleton in the 3D human skeleton model to ensure that the angle and position of each joint are consistent with the actual dance movements.

3. The dance movement auxiliary generation method based on three-dimensional modeling according to claim 1 is characterized in that: The method generates a set of motion parameters allowed for each joint in dance movements based on the three-dimensional human skeleton model and its kinematic constraints, specifically including: Based on the physiological structure and physical limitations of each joint, motion constraints are constructed to obtain degrees of freedom. Constraints include the maximum angle, minimum angle, and rotation direction of the joint, and degrees of freedom include rotational and translational degrees of freedom. A joint motion model is constructed based on constraints and degrees of freedom to obtain all possible actions of each joint and generate a set of joint actions, including discrete values ​​of joint angles, speed and acceleration limits, and timing constraints.

4. The dance movement auxiliary generation method based on three-dimensional modeling according to claim 1 is characterized in that: The dance movement auxiliary model based on the fusion of temporal convolutional network and graph convolutional neural network is constructed, and a large amount of labeled dance movement video data is input to perform dance movement auxiliary model training, which specifically includes: The dance action video data is captured continuously through a temporal convolutional network, and the three-dimensional position data of each joint in each frame is obtained. The time series characteristics of the joint movement are obtained to classify and identify the dance action categories. The adjacency relationship between joints is obtained through graph convolutional neural networks, and joints are regarded as nodes in the graph, and bone association relationships are regarded as edges of the graph; The time series features obtained by the temporal convolutional network are fused with the spatial features obtained by the graph convolutional network to construct a dance movement auxiliary model.

5. The dance movement auxiliary generation method based on three-dimensional modeling according to claim 1 is characterized in that: The dance movement auxiliary model based on the completed training inputs the three-dimensional human skeleton data of the current dance movement into the dance movement auxiliary model, combines the motion parameters allowed by each joint in the joint movement set, obtains the joint movements in the joint movement set that are associated with the current dance movement, and generates a candidate movement set, which specifically includes: Inputting the 3D human skeleton data of the current dance movement into the trained dance movement auxiliary model to obtain the joint movements associated with the current dance movement; Based on the motion parameters allowed for all joints in the joint action set, the joint action associated with the current dance action is matched with the joint actions in the joint action set to obtain a matching probability distribution; Based on the joint actions in the joint action set that are higher than a preset threshold in the matching probability distribution, a candidate action set is generated.

6. The dance movement auxiliary generation method based on three-dimensional modeling according to claim 1 is characterized in that: The method of calculating the connection smoothness score and posture beauty score between the joint movements in the candidate movement set and the current dance movement, and obtaining the candidate movement with the highest total score as the next dance movement output by weighted means specifically includes: Based on the joint movements in the candidate movement set, the difference between the joint position of the current dance movement and the position of the corresponding joint in the candidate movement is obtained, and the position difference degree is calculated; By calculating the speed of the joints in the current dance movement and the candidate movement, the speed connection fluency is calculated; The smoothness of movement transitions is scored based on positional differences and speed transition fluency; The posture aesthetics is scored based on the symmetry, balance, stability of the movement, and the stretch and tension of the posture; Based on the movement connection smoothness score and the posture beauty score, the weighted total score of each candidate movement is obtained through comprehensive weighting, and the candidate movement with the highest score is selected as the next dance movement.

7. A dance movement auxiliary generation system based on three-dimensional modeling, used to implement the dance movement auxiliary generation method based on three-dimensional modeling according to any one of claims 1 to 6, characterized in that: include: 3D human skeleton module: The 3D human skeleton module is used to extract the initial posture data of the target dance and construct a 3D human skeleton model to ensure that the angle and position of each joint are consistent with the actual dance movements; Motion parameter module: The motion parameter module generates a set of allowed motion parameters for each joint based on kinematic constraints, including joint angle, velocity, and acceleration limits; Dance movement auxiliary model module: The dance movement auxiliary model module is used to integrate the time series convolutional network and the graph convolutional neural network to analyze a large amount of labeled dance movement video data, extract time series features, and obtain the correlation between dance movements; Candidate action set module: The candidate action set module is used to input the three-dimensional human skeleton data of the current dance action, and generate a candidate action set associated with the current action in combination with the joint action set; Scoring module: The scoring module is used to calculate the connection smoothness and posture beauty scores between the candidate movements and the current dance movement, and obtain the candidate movement with the highest total score through weighting as the next dance movement output; Processor: The processor is used to process the calculation process of each formula and the construction calculation process of each model.

Citation Information

Patent Citations

  • Live video broadcasting method and device and computer storage medium

    CN112333459A

  • Dance motion scoring method and device, equipment and storage medium

    CN114333046A

  • Multi-modal feature fusion-based music dance posture generation method and device, and storage medium

    CN117316129A

  • Fine-grained dance movement scoring method based on multi-view three-dimensional human body reconstruction

    CN117496409A

  • Audio-visual dance generation method and system based on diffusion model and cross-modal collaboration

    CN118864883A

Cited By

  • AI-based dance choreography generation method, device and system

    CN120953452A

  • AI-based dance choreography generation method, device and system

    CN120953452B