Motion capture and feedback adjustment method and system for metaverse intelligent motion

Abstract

The present invention provides a motion capture and feedback adjustment method and system for metaverse intelligent sports, which relates to the field of metaverse technology. It includes collecting the user's actual motion data through multiple inertial measurement units, establishing a mapping relationship with the skeleton nodes, evaluating and optimizing the motion based on a multi-dimensional scoring matrix, constructing a user's virtual image, performing time-series analysis on the virtual motion data to obtain local and global motion features, comparing with standard motion features to generate adjustment amounts, and converting them into motion correction instructions to guide the user to adjust their posture. It can achieve accurate capture and real-time feedback, and improve the effect of sports learning.

Description

Technical Field

[0001] The present invention relates to the field of metaverse technology, and in particular to a motion capture and feedback adjustment method and system for metaverse intelligent movement. Background Art

[0002] Metaverse Intelligent Motion combines motion capture technology, avatar construction, and a motion feedback system to enable users to experience immersive movement within a virtual environment. Motion capture technology is key to achieving this motion model, typically using devices such as inertial measurement units, optical capture systems, or depth cameras to collect user motion data. In the Metaverse virtual environment, users use motion capture technology to project real-world movements into the virtual world, enabling human-computer interaction and an immersive experience.

[0003] Existing metaverse intelligent motion-related technologies have the following defects and shortcomings:

[0004] First, the motion capture accuracy in existing technologies is insufficient, especially in fast-motion scenarios, which can easily lead to skeleton point drift and position errors, resulting in a mismatch between the virtual image's movements and the user's actual movements, reducing the user's immersion and interactive experience in the metaverse environment.

[0005] Secondly, existing technologies lack effective real-time feedback and correction mechanisms. After users complete actions in a virtual environment, they often cannot obtain targeted posture adjustment suggestions in a timely manner, making it difficult to achieve self-correction and continuous improvement of actions, which affects the learning effect and training efficiency of metaverse intelligent sports. Summary of the Invention

[0006] The embodiments of the present invention provide a motion capture and feedback adjustment method and system for metaverse intelligent motion, which can solve the problems in the prior art.

[0007] A first aspect of an embodiment of the present invention provides a motion capture and feedback adjustment method for Metaverse Intelligent Movement, comprising:

[0008] The user's actual motion data during exercise is collected through multiple inertial measurement units; a mapping relationship is established between the actual motion data and preset human skeleton nodes, and the three-dimensional spatial position coordinates of each skeleton node are calculated based on the mapping relationship to obtain skeleton point position data;

[0009] A multi-dimensional scoring matrix is constructed based on the motion coherence index and the action standardization index, and each skeletal node in the skeletal point position data is evaluated and scored in real time to calculate the action score of each skeletal node; the position of the skeletal nodes in the skeletal point position data whose scores are lower than a preset scoring threshold are optimized according to the action score to generate optimized skeletal point position data;

[0010] Constructing a user virtual image using the optimized skeletal point position data, calculating virtual motion data of the user virtual image, and displaying the user virtual image in a metaverse virtual scene;

[0011] Performing a time series analysis on the virtual motion data, calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extracting the complete motion sequence using a long-time analysis method to obtain global motion features;

[0012] The local motion features and global motion features are compared with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts and displacement adjustment amounts, which are converted into motion correction instructions to guide the user to adjust the motion posture in real time.

[0013] Establishing a mapping relationship between the actual motion data and the preset human skeleton nodes, and calculating the three-dimensional spatial position coordinates of each skeleton node according to the mapping relationship to obtain the skeleton point position data includes:

[0014] The actual motion data includes acceleration data and angular velocity data obtained by a plurality of inertial measurement units; performing coordinate system alignment on the actual motion data, and converting the local coordinate systems of the plurality of inertial measurement units into a global coordinate system;

[0015] Constructing a human skeleton chain structure, the human skeleton chain structure including trunk nodes, limb nodes and joint nodes, and presetting length constraints and angle constraints between each node;

[0016] Calculating the initial positions and initial postures of the multiple inertial measurement units in the global coordinate system, and binding the initial positions and initial postures to corresponding nodes in the human skeleton chain structure; calculating the displacement change of each inertial measurement unit based on the acceleration data, and calculating the posture change of each inertial measurement unit based on the angular velocity data;

[0017] According to the displacement change and the posture change, combined with the length constraint and the angle constraint, the three-dimensional spatial position coordinates of each node in the human skeleton chain structure are calculated.

[0018] A multi-dimensional scoring matrix is constructed based on the motion coherence index and the action standardization index, and each bone node in the bone point position data is evaluated and scored in real time. The action score of each bone node is calculated including:

[0019] The skeletal point position data is sampled at fixed time intervals to generate a continuous motion data sequence, wherein the motion data sequence records the motion trajectory of each skeletal node over time; the motion data sequence is segmented to divide the motion data sequence into multiple overlapping motion segments with a preset time window length, and the overlap rate of adjacent motion segments is 50%;

[0020] Calculating motion characteristic parameters of each skeletal node in each action segment, wherein the motion characteristic parameters include position change rate, velocity change rate, and acceleration change rate, and taking a weighted sum of the position change rate, velocity change rate, and acceleration change rate as the coherence score of the action segment;

[0021] Retrieving a standard action sequence that matches the current action clip from a pre-labeled standard action library, wherein the standard action sequence includes a standard motion trajectory of each skeletal node under an ideal state;

[0022] Comparing the actual motion trajectory of each skeletal node in the current action segment with the standard motion trajectory, calculating a trajectory deviation value, and calculating a standardization score of the action segment based on the trajectory deviation value;

[0023] Mapping the coherence score and the standardization score to different dimensions of a multi-dimensional scoring matrix, respectively, setting dynamic weight coefficients for the different dimensions, and the weight coefficients are adaptively adjusted according to the action type;

[0024] The scores of each dimension in the multi-dimensional scoring matrix are multiplied by the corresponding weight coefficients and summed up to obtain the action score of each skeletal node in the current action segment.

[0025] Performing time series analysis on the virtual motion data and calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features includes:

[0026] The virtual motion data is sampled at a fixed frame rate, and N adjacent motion frames are grouped into a motion analysis window, where the number of overlapping frames between adjacent motion analysis windows is M; and the position changes of the skeletal nodes between adjacent motion frames within each motion analysis window are calculated, including spatial displacement vectors and angle change vectors.

[0027] Based on the position change, a temporal correlation matrix is constructed between the action frames, wherein the matrix elements of the temporal correlation matrix represent the motion correlation of the corresponding skeletal nodes between adjacent frames; the temporal correlation matrix is decomposed in the time domain to extract the velocity features, acceleration features and angular velocity features between adjacent action frames;

[0028] Calculate the motion consistency coefficient of each skeletal node in the motion analysis window, which is used to characterize the coherence of the local motion segment; identify abnormal motion frames in the motion analysis window based on the motion consistency coefficient, and mark motion frames with continuity feature values lower than a preset continuity threshold as abnormal frames;

[0029] Based on the velocity features, acceleration features and angular velocity features, combined with the motion consistency coefficient, a feature vector representing the local motion features is generated; the feature vector of the local motion features is associated with the feature vector of the adjacent motion analysis window to construct a complete local motion feature sequence.

[0030] The global action features obtained by extracting the complete action sequence using the long-term analysis method include:

[0031] Performing a time series analysis on the virtual motion data to obtain motion sequence data, the motion sequence data including three-dimensional coordinate information, velocity information, and acceleration information of each skeletal node from the start to the end of the motion; dividing the motion sequence data into a preparation phase, an acceleration phase, a stabilization phase, a deceleration phase, and an end phase in a temporal dimension according to a change trend of the motion state of the skeletal node;

[0032] Feature extraction is performed on the motion data of each stage, and the average velocity, maximum acceleration, displacement amplitude and angle change range of the skeletal nodes in each stage are calculated to generate the stage feature vector; the motion state changes between adjacent stages are analyzed, and the velocity continuity and position smoothness of the skeletal nodes at the stage transition moment are calculated to generate the transition feature vector;

[0033] The stage feature vector and the transformation feature vector are concatenated into a global action feature.

[0034] The method further comprises:

[0035] Constructing a two-dimensional time series feature map based on the stage feature vector and the conversion feature vector;

[0036] Performing Fourier transform on the two-dimensional time series feature graph to obtain spectrum distribution, calculating the dynamic motion energy of each skeletal node, calculating the instantaneous motion energy of each frame based on the velocity vector and acceleration vector, and generating an energy time series curve;

[0037] Aligning the energy time series curve with the two-dimensional time series feature graph in the time dimension, calculating the time series correlation between the feature intensity and the energy peak, and establishing a strength-feature mapping relationship;

[0038] Based on the force-feature mapping relationship, the coordination degree between the force change and the motion feature conversion in the action sequence is analyzed to generate a coordination evaluation index.

[0039] Comparing the local motion features and the global motion features with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts, and displacement adjustment amounts, and converting the adjustment amounts into motion correction instructions includes:

[0040] Retrieve standard action features that match the current action type from the action standard library, perform frame-level comparison between the local action features and the standard action features, calculate the posture deviation of each skeletal node between adjacent frames, and obtain real-time correction parameters; perform sequence-level comparison between the global action features and the standard action features, analyze the overall deviation trend during the action execution process, and obtain trend correction parameters;

[0041] Based on the real-time correction parameters, the angle change amount that needs to be adjusted for each skeletal node is calculated, and the angle change amount is used to correct the degree of joint bending and the direction of rotation; based on the trend correction parameters, the speed adjustment amount of the skeletal node is calculated, and the speed adjustment amount is used to optimize the movement rhythm and movement smoothness;

[0042] Calculate the displacement adjustment of the skeletal node in three-dimensional space by combining the angle change and the speed adjustment to ensure that the adjusted motion trajectory meets the standard requirements;

[0043] Prioritize the angle change, speed adjustment, and displacement adjustment, establish a multi-level correction strategy, and avoid conflicts between multiple adjustment instructions; convert the sorted adjustment amounts into specific motion correction instructions, which include adjustment direction, adjustment amplitude, and execution timing.

[0044] A second aspect of an embodiment of the present invention provides a motion capture and feedback adjustment system for Metaverse Intelligent Movement, comprising:

[0045] The first unit is configured to collect actual motion data of the user during exercise through multiple inertial measurement units; establish a mapping relationship between the actual motion data and preset human skeleton nodes, and calculate the three-dimensional spatial position coordinates of each skeleton node based on the mapping relationship to obtain skeleton point position data;

[0046] The second unit is configured to construct a multi-dimensional scoring matrix based on the motion coherence index and the action standardization index, evaluate and score each skeletal node in the skeletal point position data in real time, and calculate the action score of each skeletal node; optimize the position of the skeletal nodes in the skeletal point position data whose scores are lower than a preset scoring threshold according to the action score, and generate optimized skeletal point position data;

[0047] A third unit is configured to construct a user virtual image using the optimized skeletal point position data, calculate virtual motion data of the user virtual image, and display the user virtual image in a metaverse virtual scene;

[0048] A fourth unit is configured to perform a time series analysis on the virtual motion data, calculate the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extract the complete motion sequence using a long-time analysis method to obtain global motion features;

[0049] The fifth unit is used to compare the local motion features and global motion features with the standard motion features, generate specific angle adjustment amounts, speed adjustment amounts and displacement adjustment amounts, convert the adjustment amounts into motion correction instructions, and guide the user to adjust the motion posture in real time.

[0050] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0051] processor;

[0052] a memory for storing processor-executable instructions;

[0053] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0054] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0055] The beneficial effects of this application are as follows:

[0056] By mapping actual motion data to human skeletal nodes and calculating spatial position coordinates, the user's motion posture can be accurately captured, and real-time evaluation and optimization can be performed based on a multi-dimensional scoring matrix, which improves the accuracy and stability of motion capture and solves the problem that traditional motion capture systems are susceptible to interference in complex environments.

[0057] The optimized skeletal point position data is used to construct the user's virtual image and display it in the metaverse virtual scene, achieving a seamless connection between the physical world and the virtual world, enhancing the user's immersion and interactive experience, and making the virtual image's movements smoother and more natural, in line with the laws of human movement.

[0058] By performing time-series analysis on virtual motion data and comparing it with standard motion characteristics, specific adjustment amounts are generated and converted into motion correction instructions. This can guide users to adjust their motion posture in real time and improve exercise effects. It effectively solves the problem of traditional sports teaching that is difficult to accurately guide users' motion details, and provides users with personalized sports guidance and feedback. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1Schematic diagram of the flow of a motion capture and feedback adjustment method for Metaverse Intelligent Movement according to an embodiment of the present invention;

[0060] Figure 2 Schematic diagram of the flow of generating motion correction instructions. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0062] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0063] refer to Figure 1 and Figure 2 The embodiment of the present invention is a method for motion capture and feedback adjustment for Metaverse Intelligent Movement, comprising:

[0064] The user's actual motion data during exercise is collected through multiple inertial measurement units; a mapping relationship is established between the actual motion data and preset human skeleton nodes, and the three-dimensional spatial position coordinates of each skeleton node are calculated based on the mapping relationship to obtain skeleton point position data;

[0065] A multi-dimensional scoring matrix is constructed based on the motion coherence index and the action standardization index, and each skeletal node in the skeletal point position data is evaluated and scored in real time to calculate the action score of each skeletal node; the position of the skeletal nodes in the skeletal point position data whose scores are lower than a preset scoring threshold are optimized according to the action score to generate optimized skeletal point position data;

[0066] Constructing a user virtual image using the optimized skeletal point position data, calculating virtual motion data of the user virtual image, and displaying the user virtual image in a metaverse virtual scene;

[0067] Performing a time series analysis on the virtual motion data, calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extracting the complete motion sequence using a long-time analysis method to obtain global motion features;

[0068] The local motion features and global motion features are compared with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts and displacement adjustment amounts, which are converted into motion correction instructions to guide the user to adjust the motion posture in real time.

[0069] In an optional embodiment, a mapping relationship is established between the actual motion data and preset human skeleton nodes, and the three-dimensional spatial position coordinates of each skeleton node are calculated according to the mapping relationship to obtain the skeleton point position data, including:

[0070] The actual motion data includes acceleration data and angular velocity data obtained by a plurality of inertial measurement units; performing coordinate system alignment on the actual motion data, and converting the local coordinate systems of the plurality of inertial measurement units into a global coordinate system;

[0071] Constructing a human skeleton chain structure, the human skeleton chain structure including trunk nodes, limb nodes and joint nodes, and presetting length constraints and angle constraints between each node;

[0072] Calculating the initial positions and initial postures of the multiple inertial measurement units in the global coordinate system, and binding the initial positions and initial postures to corresponding nodes in the human skeleton chain structure; calculating the displacement change of each inertial measurement unit based on the acceleration data, and calculating the posture change of each inertial measurement unit based on the angular velocity data;

[0073] According to the displacement change and the posture change, combined with the length constraint and the angle constraint, the three-dimensional spatial position coordinates of each node in the human skeleton chain structure are calculated.

[0074] For example, the collected raw data consists of two parts: acceleration data, represented as ax, ay, az, and in units of m / s²; and angular velocity data, represented as wx, wy, and wz, and in units of rad / s. Since each IMU has its own local coordinate system, coordinate system alignment is first required. The specific implementation process is as follows:

[0075] The Earth coordinate system is selected as the global reference coordinate system, where the direction of gravity is the negative Z-axis, the front of the body is the positive Y-axis, and the right side is the positive X-axis. For each inertial measurement unit (IMU), an initial attitude calibration is performed in advance, and the initial quaternion q_initial is recorded. This quaternion describes the rotation relationship between the IMU's local coordinate system and the global coordinate system. All subsequent measurement data is transformed using this quaternion, converting the acceleration and angular velocity in the local coordinate system to the global coordinate system. For example, if an IMU measures an acceleration of (2.1, -0.5, 9.8) m / s² in the local coordinate system, the data obtained in the global coordinate system after the quaternion rotation transformation may be (0.2, 1.8, 9.6) m / s².

[0076] Next, we build a human skeleton chain structure model. This model consists of multiple nodes and bone segments connecting the nodes. Based on standard human anatomical features, we define the following node types:

[0077] 1. Trunk nodes: including the center of the head, neck, chest, waist and pelvis;

[0078] 2. Limb joints: including left / right shoulder, left / right elbow, left / right wrist, left / right hip joint, left / right knee, left / right ankle;

[0079] 3. Joint nodes: The key points connecting the above nodes, such as shoulder joints, elbow joints, etc.

[0080] At the same time, in order to make the reconstructed human skeleton conform to physiological characteristics, two types of constraints are preset:

[0081] - Length Constraint: Determine the length of each bone segment based on the subject's body data. For example, for an adult male with a height of 175 cm, the upper arm length is approximately 28 cm and the forearm length is approximately 26 cm.

[0082] - Angle Constraint: Set angle limits based on the range of motion of human joints. For example, the flexion and extension angle range of the elbow joint is 0°-145°, and the abduction angle range of the shoulder joint is 0°-180°.

[0083] Determining the initial pose is a crucial step in the capture process. The subject is asked to stand with their arms hanging naturally, facing forward. Approximately three seconds of static data are collected during this time to determine the initial position and pose of each IMU in the global coordinate system. Vertical orientation is determined using gravity alignment, and forward orientation is determined using a predefined forward pose. For example, the initial position of the chest IMU might be (0, 0, 0.7) m, indicating a height of 0.7 meters relative to the ground coordinate origin.

[0084] When binding an IMU to a skeletal node, establish a corresponding relationship based on the IMU's actual placement on the body. For example, the right upper arm IMU is associated with the right shoulder and right elbow nodes, and the waist IMU is associated with the lumbar vertebrae node. Create a mapping table to record the relative position of each IMU and its corresponding skeletal node.

[0085] The core of motion capture is to update the position of skeleton nodes in real time. For each frame of data (assuming the time interval is dt, a typical value is 16.7ms), the following processing is performed:

[0086] First, the acceleration data is processed to remove the effects of gravity and obtain the linear acceleration generated by human motion. Displacement is calculated using a quadratic integration method. The specific steps are: based on the current frame acceleration a and the previous frame velocity v, calculate the current frame velocity v' = v + a × dt; then, based on the average velocity, calculate the displacement change ds = (v + v') / 2 × dt. For example, if an IMU measures a horizontal acceleration of 2 m / s² with a time interval of 16.7 ms, the single-frame velocity increment is approximately 0.033 m / s, corresponding to a displacement change of approximately 0.28 mm.

[0087] When processing angular velocity data, the attitude quaternion is updated by integrating the angular velocity. For example, if the IMU measures an angular velocity of (0.1, 0.2, -0.3) rad / s with a time interval of 16.7ms, the rotation increment can be calculated and applied to the current attitude quaternion to obtain the updated attitude.

[0088] To address the problem of cumulative integral errors, this method uses sliding window filtering and zero-velocity update techniques for correction. When the IMU is detected as being approximately stationary (acceleration and angular velocity changes are less than preset thresholds, such as 0.1 m / s² and 0.05 rad / s), the IMU velocity is considered zero and the integral calculation is reset.

[0089] Finally, based on the updated position and posture, combined with the skeletal constraints, an inverse kinematics algorithm is used to calculate the 3D spatial position of each node in the skeletal chain. When multiple IMUs influence the same skeletal node, a confidence-based weighted average method is used to determine the final position. For example, if the elbow joint position is influenced by both the upper arm IMU and the forearm IMU, a weight can be assigned based on the measurement confidence of each. For example, if the weights are 0.6 and 0.4, respectively, the final position is the weighted average of the two calculations.

[0090] Through the above-mentioned technical implementation, this method can achieve accurate human motion capture, and the accuracy of skeleton point position can reach centimeter level, which is suitable for various application scenarios such as virtual reality, motion analysis, rehabilitation training, etc.

[0091] In an optional embodiment, a multi-dimensional scoring matrix is constructed based on the motion coherence index and the action standardization index, and each bone node in the bone point position data is evaluated and scored in real time. Calculating the action score of each bone node includes:

[0092] The skeletal point position data is sampled at fixed time intervals to generate a continuous motion data sequence, wherein the motion data sequence records the motion trajectory of each skeletal node over time; the motion data sequence is segmented to divide the motion data sequence into multiple overlapping motion segments with a preset time window length, and the overlap rate of adjacent motion segments is 50%;

[0093] Calculating motion characteristic parameters of each skeletal node in each action segment, wherein the motion characteristic parameters include position change rate, velocity change rate, and acceleration change rate, and taking a weighted sum of the position change rate, velocity change rate, and acceleration change rate as the coherence score of the action segment;

[0094] Retrieving a standard action sequence that matches the current action clip from a pre-labeled standard action library, wherein the standard action sequence includes a standard motion trajectory of each skeletal node under an ideal state;

[0095] Comparing the actual motion trajectory of each skeletal node in the current action segment with the standard motion trajectory, calculating a trajectory deviation value, and calculating a standardization score of the action segment based on the trajectory deviation value;

[0096] Mapping the coherence score and the standardization score to different dimensions of a multi-dimensional scoring matrix, respectively, setting dynamic weight coefficients for the different dimensions, and the weight coefficients are adaptively adjusted according to the action type;

[0097] The scores of each dimension in the multi-dimensional scoring matrix are multiplied by the corresponding weight coefficients and summed up to obtain the action score of each skeletal node in the current action segment.

[0098] This embodiment provides a method for constructing a multi-dimensional scoring matrix based on motion coherence indicators and action standardization indicators, and performs real-time evaluation and scoring of each bone node in the bone point position data.

[0099] First, the system uses sensors or motion capture devices to obtain skeletal position data as the user performs actions. This includes at least 15 key skeletal nodes, such as the head, shoulders, elbows, wrists, hips, knees, and ankles. The position of each skeletal node is represented by three-dimensional coordinates (x, y, z).

[0100] The system samples skeletal point position data at regular intervals, for example, every 20 milliseconds, generating a continuous sequence of motion data. For example, if a user performs a five-second squat, the system will collect approximately 250 frames of skeletal point position data, forming a complete motion sequence.

[0101] Next, the system segments the action data sequence. A preset time window length, such as 2 seconds (i.e. 100 frames), is used as a complete action segment, and the overlap rate of adjacent action segments is set to 50%. Therefore, the first action segment contains data from frames 1-100, the second action segment contains data from frames 51-150, and so on. Through this overlapping segmentation method, the system can capture the continuous changes in the action and ensure that key action frames are not missed.

[0102] For each action segment, the system calculates the motion characteristic parameters of each skeletal node. Taking the right knee joint as an example, the system first calculates the position change rate: the system extracts the position change of the right knee joint between each two adjacent frames in the action segment, accumulates the position change between all frames, and divides it by the total number of frames to obtain the average position change rate. For example, the average position change rate of the right knee joint in one action segment is 0.05 meters per frame.

[0103] The velocity change rate is calculated by first calculating the instantaneous velocity between each two adjacent frames, then calculating the difference between the two instantaneous velocities (the velocity delta). The sum of all velocity deltas is then divided by the total number of frames minus 2 to obtain the average velocity change rate. For example, the average velocity change rate of the right knee joint in the same action clip is 0.003 m / frame².

[0104] Similarly, the acceleration rate is calculated by taking the difference between two adjacent acceleration values and averaging them. For example, the average acceleration rate of the right knee joint in this action clip is 0.0005 m / frame³.

[0105] The system assigns different weights to calculate the coherence score. For example, the weight for position change rate is 0.3, the weight for velocity change rate is 0.4, and the weight for acceleration change rate is 0.3. For the right knee joint example above, the coherence score is: 0.05 × 0.3 + 0.003 × 0.4 + 0.0005 × 0.3 = 0.01665. The score range is normalized to a 0-100 scale.

[0106] At the same time, the system retrieves matching standard movement sequences from a pre-annotated library of standard movements. This library includes a variety of common movement types, such as squats, lunges, and planks. Each movement has been demonstrated by professional athletes and manually annotated. The system uses a dynamic time warping algorithm to compare the current movement clip with the standard movement sequence and selects the standard movement with the highest similarity as a reference.

[0107] The system compares the actual motion trajectory of each skeletal node in the current action clip with the standard motion trajectory. For example, the system calculates the Euclidean distance between the actual position and the standard position in each frame. This distance is then accumulated across all frames and divided by the total number of frames to obtain the average trajectory deviation. For example, the average trajectory deviation for the right knee in one action clip is 0.08 meters.

[0108] The standardization score is calculated using an inverse proportional relationship: smaller deviations correspond to higher scores. The specific calculation method is: Standardization score = 100 - Deviation value × coefficient, where the coefficient is adjusted based on the importance of different skeletal nodes. For example, if the importance coefficient of the knee joint in the squat is 1.5, the standardization score for the right knee joint is 100 - 0.08 × 1.5 × 100 = 88 points.

[0109] The system constructs a multidimensional scoring matrix, mapping the coherence and standardization scores to different dimensions. For squats, the standardization dimension is typically weighted more highly, for example, with a coherence weight of 0.4 and a standardization weight of 0.6. The system adaptively adjusts these weights based on the type of movement; for example, for dance-style movements, the coherence weight might be increased to 0.6.

[0110] Finally, the system multiplies the scores of each dimension in the multidimensional scoring matrix by the corresponding weights and sums them to obtain a comprehensive score for each skeletal node in the current action segment. Taking the right knee joint as an example, if its coherence score (normalized) is 85 and its standardization score is 88, the comprehensive score is 85 × 0.4 + 88 × 0.6 = 86.8.

[0111] The system also sets specific scoring priorities for different types of movements. For example, in a squat, the system will give higher weight to the knee and hip joints, which may account for 60% of the total score; while in a plank, the system will pay more attention to the stability of the spine and shoulders, and the score weight of the bone nodes in these parts can reach 65%.

[0112] Through this multi-dimensional, adaptive weight scoring mechanism, the system can comprehensively and objectively evaluate the quality of user actions and provide targeted improvement suggestions.

[0113] In an optional embodiment, performing a time series analysis on the virtual motion data and calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features includes:

[0114] The virtual motion data is sampled at a fixed frame rate, and N adjacent motion frames are grouped into a motion analysis window, where the number of overlapping frames between adjacent motion analysis windows is M; and the position changes of the skeletal nodes between adjacent motion frames within each motion analysis window are calculated, including spatial displacement vectors and angle change vectors.

[0115] Based on the position change, a temporal correlation matrix is constructed between the action frames, wherein the matrix elements of the temporal correlation matrix represent the motion correlation of the corresponding skeletal nodes between adjacent frames; the temporal correlation matrix is decomposed in the time domain to extract the velocity features, acceleration features and angular velocity features between adjacent action frames;

[0116] Calculate the motion consistency coefficient of each skeletal node in the motion analysis window, which is used to characterize the coherence of the local motion segment; identify abnormal motion frames in the motion analysis window based on the motion consistency coefficient, and mark motion frames with continuity feature values lower than a preset continuity threshold as abnormal frames;

[0117] Based on the velocity features, acceleration features and angular velocity features, combined with the motion consistency coefficient, a feature vector representing the local motion features is generated; the feature vector of the local motion features is associated with the feature vector of the adjacent motion analysis window to construct a complete local motion feature sequence.

[0118] The virtual action data is sampled at a fixed frame rate. In this embodiment, the action data of the virtual character is sampled at a fixed frame rate of 30 frames per second. N adjacent action frames are combined into a motion analysis window, and the number of overlapping frames between adjacent action analysis windows is M. Specifically, N = 60 frames can be set, that is, each action analysis window contains 2 seconds of action data; M = 15 frames can be set, that is, the action data overlaps 0.5 seconds between adjacent windows. This overlapping window design can ensure the continuity of motion analysis and avoid analysis faults at the window boundaries.

[0119] Assuming the avatar's skeletal model contains K nodes (for example, K=23, including key nodes such as the head, neck, shoulders, elbows, wrists, chest, waist, hips, knees, and ankles), for each skeletal node j, the spatial displacement vector ΔP(i, j) and angular change vector ΔA(i, j) are calculated between the i-th frame and the i+1-th frame. The spatial displacement vector represents the change in the skeletal node's position in three-dimensional space and consists of three components: x, y, and z. The angular change vector represents the change in the Euler angle of the skeletal node relative to its parent node, consisting of the change in rotation angle around the x-, y-, and z-axes.

[0120] For example, for the change of the right elbow node between the 10th frame and the 11th frame, the possible spatial displacement vector is (0.05 meters, 0.02 meters, -0.03 meters), and the angle change vector is (2 degrees, -1 degrees, 0.5 degrees).

[0121] The temporal correlation matrix C between action frames is constructed based on the position change. For a window consisting of N action frames, the size of the temporal correlation matrix C is (N-1)×K×K. The matrix elements C(i,j, l ) indicates that between the i-th frame and the i+1-th frame, the bone node j and the bone node l The correlation is calculated by comparing the motion of node j with that of node l The similarity between the displacement vector and the angle change vector is used to determine the movement trend of the two nodes. A high similarity indicates that the movement trends of the two nodes are consistent, while a low similarity indicates that the movement trends are different.

[0122] The temporal correlation matrix is decomposed in the time domain to extract velocity, acceleration, and angular velocity features between adjacent action frames. The velocity feature V(i,j) is calculated by dividing the displacement vector ΔP(i,j) by the inter-frame time interval (e.g., 1 / 30 second). The acceleration feature A(i,j) is calculated by dividing the velocity difference between two adjacent frames by the inter-frame time interval. The angular velocity feature ω(i,j) is calculated by dividing the angle change vector ΔA(i,j) by the inter-frame time interval.

[0123] In a real-world case, the velocity characteristics of the right elbow node at a given moment might be (1.5 m / s, 0.6 m / s, -0.9 m / s), the acceleration characteristics might be (0.3 m / s², -0.1 m / s², 0.2 m / s²), and the angular velocity characteristics might be (60 degrees / s, -30 degrees / s, 15 degrees / s).

[0124] Next, the motion consistency coefficient R of each skeletal node in the motion analysis window is calculated. The motion consistency coefficient is used to characterize the coherence of the local motion segment. It is calculated by analyzing the stability of the changes in the velocity direction and acceleration direction within the window. Specifically, for skeletal node j, its motion consistency coefficient R(j) can be represented by the average cosine value of the angle between the velocity vectors of adjacent frames in the window. The closer the R(j) value is to 1, the more consistent the motion direction of the node in the window and the more coherent the motion; the closer the R(j) value is to 0 or negative, the more drastic the change in motion direction and the possibility of incoherent motion.

[0125] Abnormal motion frames within the motion analysis window are identified based on the motion consistency coefficient. A preset continuity threshold T = 0.65 is set. When the motion consistency coefficient R(j) of any key skeletal node j in a frame i falls below T, the frame is marked as abnormal. Abnormal frames may indicate sudden changes in motion, jitter, or unnatural transitions, requiring special attention or correction in subsequent processing.

[0126] In one specific example, analyzing windowed data from a running motion revealed that the left knee joint motion consistency coefficient for frame 35 was only 0.42, below the threshold of 0.65, and the system flagged it as an abnormal frame. Further inspection revealed that the left knee joint angle in this frame had suddenly changed abnormally, inconsistent with a normal running pattern.

[0127] Based on the extracted velocity, acceleration, and angular velocity features, combined with the motion consistency coefficient, a feature vector F is generated to characterize the local motion. For each motion analysis window, feature vector F contains the average velocity, maximum velocity, average acceleration, maximum acceleration, average angular velocity, and maximum angular velocity of all skeletal nodes within the window, as well as the motion consistency coefficient for each node. For example, for a character with 23 skeletal nodes, feature vector F can have hundreds of dimensions, fully describing the motion characteristics within the window.

[0128] The feature vectors of the local motion features are correlated with the feature vectors of adjacent motion analysis windows to construct a complete local motion feature sequence S. Because adjacent windows overlap, the features in these overlapping regions need to be smoothly fused. This can be achieved using a weighted average, with the weights dependent on the position of the samples in the overlapping region. This allows the resulting feature sequence S to coherently represent the local dynamic characteristics of the entire virtual motion.

[0129] In an optional embodiment, extracting the complete action sequence using a long-term analysis method to obtain global action features includes:

[0130] Performing a time series analysis on the virtual motion data to obtain motion sequence data, the motion sequence data including three-dimensional coordinate information, velocity information, and acceleration information of each skeletal node from the start to the end of the motion; dividing the motion sequence data into a preparation phase, an acceleration phase, a stabilization phase, a deceleration phase, and an end phase in a temporal dimension according to a change trend of the motion state of the skeletal node;

[0131] Feature extraction is performed on the motion data of each stage, and the average velocity, maximum acceleration, displacement amplitude and angle change range of the skeletal nodes in each stage are calculated to generate the stage feature vector; the motion state changes between adjacent stages are analyzed, and the velocity continuity and position smoothness of the skeletal nodes at the stage transition moment are calculated to generate the transition feature vector;

[0132] The stage feature vector and the transformation feature vector are concatenated into a global action feature.

[0133] In this embodiment, the global features of the complete action sequence are extracted through long time series analysis, which mainly includes steps such as action sequence data processing, action stage division, feature extraction and global feature construction.

[0134] The virtual motion data is time-series analyzed to obtain motion sequence data. This embodiment uses a depth camera to collect human motion information and extracts 20 key skeletal nodes through a human skeleton recognition algorithm, including locations such as the head, neck, shoulders, elbows, wrists, spine, hips, knees, and ankles. The system records the three-dimensional coordinates (x, y, z) of each skeletal node at a sampling rate of 30 frames per second to form an original coordinate sequence. For a specific skeletal node, such as the right wrist, its coordinates at time t are marked as (x_t, y_t, z_t).

[0135] To calculate velocity information, the system performs a differential calculation on the coordinate changes between two adjacent frames. For example, the velocity v_t of the right wrist at time t is calculated as the difference between the coordinates of the current and previous frames divided by the sampling interval. Specifically, if the coordinates of the right wrist in frame 10 are (0.45m, 1.2m, 0.3m), and in frame 11 are (0.48m, 1.25m, 0.32m), and the sampling interval is 0.033 seconds, the velocity is calculated as ((0.48-0.45) / 0.033, (1.25-1.2) / 0.033, (0.32-0.3) / 0.033) = (0.91m / s, 1.52m / s, 0.61m / s).

[0136] Acceleration information is obtained by performing a quadratic difference on the velocity sequence. For example, the acceleration of the right wrist at time t, a_t, is the difference between the current velocity and the previous velocity divided by the sampling interval. Based on the above example, if the velocities calculated from frames 9 to 10 are (0.76 m / s, 1.21 m / s, 0.45 m / s), then the acceleration at frame 10 is ((0.91 - 0.76) / 0.033, (1.52 - 1.21) / 0.033, (0.61 - 0.45) / 0.033) = (4.55 m / s², 9.39 m / s², 4.85 m / s²).

[0137] Based on the changing trends of the motion states of the skeletal nodes, the motion sequence data is divided into five stages along the time dimension. First, the velocity curve of each node is calculated, and then the stages are divided by setting speed thresholds. In this embodiment, for the throwing action, the speed thresholds are set at 15%, 40%, 60%, and 30% of the maximum speed, respectively, as the dividing points of each stage.

[0138] The preparation phase is defined as the period from the start of the movement to the first time the speed exceeds 15% of the maximum speed. For example, if the maximum speed of the right wrist in a certain movement sequence is 3.6 m / s, the preparation phase ends when the speed first exceeds 0.54 m / s. In this case, the preparation phase lasts from frame 1 to frame 18, and the right wrist mainly completes the transition from static to initial motion.

[0139] The acceleration phase lasts from the end of the preparation phase until the speed reaches 40% of its maximum value. In the example above, the acceleration phase ends when the speed reaches 1.44 m / s, corresponding to frame 32 in the sequence. During this phase, the speed of the right wrist increases from 0.54 m / s to 1.44 m / s, with a displacement of approximately 12 cm.

[0140] The stabilization phase, from the end of the acceleration phase to the time when the speed reaches 60% of its maximum value, corresponds to a speed increase from 1.44 m / s to 2.16 m / s, occurring between frames 32 and 45. During this phase, the right wrist displaced approximately 25 cm and the angle changed by approximately 15 degrees.

[0141] The deceleration phase lasts from the end of the stabilization phase to the time when the speed drops to 30% of the maximum speed, that is, from frames 45 to 68, the speed drops from 2.16 m / s to 1.08 m / s. During this phase, the right wrist displaces approximately 30 cm, and the acceleration changes significantly.

[0142] The final phase, from the end of the deceleration phase to the complete cessation of the movement, occurs from frames 68 to 85, with the speed decreasing from 1.08 m / s to nearly 0 m / s. During this phase, the displacement is approximately 8 cm, and the movement gradually stops and stabilizes.

[0143] Feature extraction was performed on the motion data for each phase. The average speed of the skeletal nodes in each phase was calculated. For example, the average speed of the right wrist was 0.32 m / s in the preparation phase, 0.92 m / s in the acceleration phase, 1.75 m / s in the stabilization phase, 1.58 m / s in the deceleration phase, and 0.42 m / s in the final phase.

[0144] The maximum acceleration of each stage was calculated. The maximum acceleration of the right wrist was 2.7 m / s² in the preparation stage, 8.5 m / s² in the acceleration stage, 6.2 m / s² in the stabilization stage, -9.1 m / s² in the deceleration stage, and -4.3 m / s² in the end stage.

[0145] Calculate the displacement amplitude. Taking the right wrist as an example, the displacement amplitude in the preparation phase is 5 cm, in the acceleration phase is 12 cm, in the stabilization phase is 25 cm, in the deceleration phase is 30 cm, and in the end phase is 8 cm.

[0146] The range of angle change was calculated. The range of elbow joint angle change in the preparation phase was 5 degrees to 25 degrees, in the acceleration phase was 25 degrees to 65 degrees, in the stabilization phase was 65 degrees to 80 degrees, in the deceleration phase was 80 degrees to 45 degrees, and in the ending phase was 45 degrees to 30 degrees.

[0147] The above calculation results form the eigenvectors for each stage. For example, the eigenvector for the preparation stage is [0.32m / s, 2.7m / s², 5cm, 20 degrees]. The remaining stages are constructed similarly.

[0148] We analyze the changes in motion between adjacent phases and calculate the velocity continuity and position smoothness of skeletal nodes at phase transitions to generate transition feature vectors. Velocity continuity is calculated by averaging the velocity differences over the three frames before and after the transition between adjacent phases. For example, at the transition from the preparation phase to the acceleration phase, the velocity continuity value is 0.08 m / s, indicating a smooth transition.

[0149] Position smoothness is calculated by calculating the jitter of the position coordinates three frames before and after the transition point. For example, a position smoothness value of 0.5 cm at the transition point between the preparation phase and the acceleration phase indicates stable position change. Example feature vectors for each transition point include: the preparation-acceleration transition point [0.08 m / s, 0.5 cm], the acceleration-stabilization transition point [0.12 m / s, 0.8 cm], the stabilization-deceleration transition point [0.15 m / s, 1.2 cm], and the deceleration-end transition point [0.07 m / s, 0.6 cm].

[0150] Finally, the stage feature vector and the transformation feature vector are concatenated to form the global action feature. For the above throwing action example, the global action feature is [0.32m / s, 2.7m / s², 5cm, 20 degrees, 0.92m / s, 8.5m / s², 12cm, 40 degrees, 1.75m / s, 6.2m / s², 25cm, 15 degrees, 1.58m / s, -9.1m / s², 30cm, 35 degrees, 0.42m / s, -4.3m / s², 8cm, 15 degrees, 0.08m / s, 0.5cm, 0.12m / s, 0.8cm, 0.15m / s, 1.2cm, 0.07m / s,0.6cm], a total of 28-dimensional feature vectors, which comprehensively describe the timing characteristics and stage transition characteristics of the entire action.

[0151] This global motion feature can be used in application scenarios such as motion recognition, motion quality assessment, and motion similarity comparison, providing a more comprehensive motion timing representation than single-frame features.

[0152] In an optional embodiment, the method further includes:

[0153] Constructing a two-dimensional time series feature map based on the stage feature vector and the conversion feature vector;

[0154] Performing Fourier transform on the two-dimensional time series feature graph to obtain spectrum distribution, calculating the dynamic motion energy of each skeletal node, calculating the instantaneous motion energy of each frame based on the velocity vector and acceleration vector, and generating an energy time series curve;

[0155] Aligning the energy time series curve with the two-dimensional time series feature graph in the time dimension, calculating the time series correlation between the feature intensity and the energy peak, and establishing a strength-feature mapping relationship;

[0156] Based on the force-feature mapping relationship, the coordination degree between the force change and the motion feature conversion in the action sequence is analyzed to generate a coordination evaluation index.

[0157] This embodiment provides a method for evaluating movement coordination based on time series characteristics and dynamic energy analysis. This method constructs a two-dimensional time series characteristic graph, performs spectrum analysis, establishes a force-feature mapping relationship, and generates a coordination evaluation index to objectively evaluate the coordination of human movement sequences. The following are detailed technical implementation steps:

[0158] Based on the acquired stage feature vectors and transformation feature vectors, a two-dimensional temporal feature map is constructed. The horizontal axis of the feature map represents the time dimension, which specifically corresponds to the frame index or timestamp of the action sequence; the vertical axis represents the feature dimension, which corresponds to different feature components; and the pixel value in the map represents the feature intensity. In specific implementation, the feature vectors at each time point are arranged in rows to form a matrix structure. For example, for an action sequence with a length of 120 frames, the feature dimension is 64, and the size of the constructed two-dimensional temporal feature map is 64×120. To enhance the visualization effect, a heat map can be used for presentation, where areas with high feature intensity are displayed in warm tones (such as red) and areas with low feature intensity are displayed in cool tones (such as blue).

[0159] Perform a Fourier transform on the constructed two-dimensional time series feature graph to obtain the spectrum distribution. The specific steps are: perform a fast Fourier transform on the time series of each feature dimension separately to obtain the corresponding frequency components and amplitudes; summarize the spectrum results of all feature dimensions to form a spectrum distribution matrix. For example, for a 64-dimensional feature, 64 spectrum distribution curves can be obtained. By analyzing the spectrum distribution, the main periodic patterns in the movement can be identified. In practice, taking dance movements as an example, if the spectrum has a clear peak in the 2-4 Hz range, it means that the movement is repeated at a frequency of 2-4 times per second.

[0160] The three-dimensional coordinate sequence of 25 key skeletal nodes is extracted from the motion capture data; the position difference between adjacent frames is calculated to obtain the velocity vector; the difference of the velocity vector is further calculated to obtain the acceleration vector. Based on the velocity vector and the acceleration vector, the instantaneous motion energy of each frame is calculated. The specific calculation method is: for the t-th frame, first calculate the weighted sum of the squared velocity and the squared acceleration of all skeletal nodes as the instantaneous energy value E(t) of the frame. The weight can be set according to the importance of the skeletal node. Usually, the weight of the torso is higher (such as 0.4), and the weight of the extremities is lower (such as 0.2). The instantaneous energy values of all frames are connected to form an energy time series curve.

[0161] For example, the energy curve of a 120-frame modern dance sequence performed by a professional dancer exhibits distinct fluctuations, with energy peaks occurring at transition points and moments of intense power. Specifically, the data shows that static postures have an energy value of approximately 5-10 units, moderate-intensity movements have an energy value of approximately 30-50 units, and high-intensity movements like jumps have energy peaks of up to 80-100 units.

[0162] The energy time series curve is aligned with the two-dimensional time series feature map in the time dimension, and the time series correlation between the feature intensity and the energy peak is calculated to establish a force-feature mapping relationship. The specific implementation is as follows: the energy curve is normalized so that its value range is between 0 and 1; the time series data of each feature dimension in the feature map is also normalized; the cross-correlation function between the energy curve and the time series data of each feature dimension is calculated to determine the time delay relationship and correlation coefficient; feature dimensions with correlation coefficients exceeding a threshold (such as 0.7) are screened, as these dimensions are considered to be closely related to the change in action force; these highly correlated feature dimensions are paired with the corresponding energy change patterns to form a force-feature mapping table.

[0163] Taking the aforementioned modern dance performance as an example, analysis shows that the correlation coefficients between the 3rd, 7th, 15th, and 22nd dimension features and the energy curve are 0.81, 0.75, 0.89, and 0.72, respectively. These features primarily correspond to the characteristic expressions of trunk twisting, arm extension, leg flexion and extension, and jumping movements. The force-feature mapping table shows that when the energy value is in the range of 75-85, the 15th dimension feature value should be in the range of 0.8-0.9, indicating a typical characteristic response for high-intensity leg movements.

[0164] Based on the force-feature mapping relationship, the coordination between force changes and motion feature conversion in an action sequence is analyzed to generate a coordination evaluation index. The specific method is as follows: for each frame, the ideal feature response value is retrieved from the force-feature mapping table based on the current energy value; the Euclidean distance between the actual feature value and the ideal value is calculated as the coordination deviation of the single frame; the coordination deviations of the entire sequence are averaged to obtain the average coordination deviation index; and the average deviation value is converted into a coordination score on a scale of 0-100, with smaller deviations indicating higher scores.

[0165] In practical applications, a comparative analysis of the same dance moves performed by professional and novice dancers revealed that the average coordination deviation for the professionals was 0.12, translating to a coordination score of 92; while the average deviation for the novices was 0.31, resulting in a score of 73. This demonstrates that the method can effectively differentiate the movement coordination of performers of different skill levels, providing an objective quantitative basis for movement quality assessment.

[0166] Through the above method, not only can the overall coordination be evaluated, but also the specific incoordination time periods and corresponding movement characteristics can be located, providing accurate feedback for movement training.

[0167] In an optional embodiment, comparing the local motion features and the global motion features with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts, and displacement adjustment amounts, and converting the adjustment amounts into motion correction instructions includes:

[0168] Retrieve standard action features that match the current action type from the action standard library, perform frame-level comparison between the local action features and the standard action features, calculate the posture deviation of each skeletal node between adjacent frames, and obtain real-time correction parameters; perform sequence-level comparison between the global action features and the standard action features, analyze the overall deviation trend during the action execution process, and obtain trend correction parameters;

[0169] Based on the real-time correction parameters, the angle change amount that needs to be adjusted for each skeletal node is calculated, and the angle change amount is used to correct the degree of joint bending and the direction of rotation; based on the trend correction parameters, the speed adjustment amount of the skeletal node is calculated, and the speed adjustment amount is used to optimize the movement rhythm and movement smoothness;

[0170] Calculate the displacement adjustment of the skeletal node in three-dimensional space by combining the angle change and the speed adjustment to ensure that the adjusted motion trajectory meets the standard requirements;

[0171] Prioritize the angle change, speed adjustment, and displacement adjustment, establish a multi-level correction strategy, and avoid conflicts between multiple adjustment instructions; convert the sorted adjustment amounts into specific motion correction instructions, which include adjustment direction, adjustment amplitude, and execution timing.

[0172] The system retrieves standard action features that match the current action type from the action standard library. The system maintains an action standard library that stores feature data for various standard actions. For example, for the yoga tree pose, the standard library stores key skeletal node position data for the pose, including information such as the hip joint angle should be 90°±5° when the left leg is raised, the knee extension of the supporting leg should be greater than 175°, and the angle between the arms and the shoulders should be 180°±10° when the arms are raised. When the system detects that the user is performing the tree pose, it matches the corresponding standard action features through the action recognition algorithm.

[0173] The system compares local motion features with standard motion features at the frame level. The system collects data for each frame of the user's current motion and compares and analyzes the posture of each skeletal node between adjacent frames. Specifically, taking the hip joint as an example, if the standard tree pose requires a hip joint angle of 90°, and the user's actual angle is 75°, the system records the joint angle deviation as 15°. The system performs similar deviation calculations on the 20 main joints of the body and saves these deviation data as real-time correction parameters. This parameter set contains data such as the angle difference, position offset, and rotation error of each joint.

[0174] The system compares global motion features with standard motion features at the sequence level. The system collects 30 consecutive frames (approximately 1 second) of motion data from the user and analyzes their overall motion trends. For example, when analyzing a user completing a squat, the system detects that the user's squat speed gradually slows down from 0.3m / s at the beginning to 0.1m / s, while the standard squat requires a uniform squat speed of 0.25m / s. The system records this speed change trend as a trend correction parameter for subsequent optimization of the movement rhythm.

[0175] Based on the real-time correction parameters, the system calculates the angle changes that need to be adjusted for each skeletal node. Taking the arm raising action as an example, if the user's elbow joint is bent at 150°, and the standard action requires 170°, the system generates an angle adjustment of 20°. For joints with multi-dimensional rotation, such as the shoulder joint, the system calculates the angle adjustments for the three degrees of freedom separately. For example, the forward direction needs to increase by 15°, the abduction direction needs to decrease by 8°, and the internal rotation needs to decrease by 10°. These angle changes are directly used to correct the degree of joint bending and the direction of rotation.

[0176] Based on the trend correction parameters, the system calculates the speed adjustment for the skeletal nodes. For example, for running, if the standard movement requires an arm swing frequency of 1.2 times per second, but the user's actual frequency is 0.8 times per second, the system generates a speed adjustment instruction to increase the arm swing speed by 50%. The system also analyzes the smoothness of the transitions between movements and generates corresponding speed adjustments for any pauses to ensure movement continuity.

[0177] The system combines angle changes and speed adjustments to calculate the displacement adjustments of skeletal nodes in three-dimensional space. For example, during a squat, if the user's knee angle needs to be adjusted forward by 15° and their speed needs to be increased by 20%, the combined calculation shows that the required displacement adjustment of the knee center point in the forward direction is 7 cm and in the upward direction is 5 cm. The system performs similar displacement calculations for key nodes throughout the body to ensure that the adjusted motion trajectory meets standard requirements.

[0178] The system prioritizes angle changes, speed adjustments, and displacement adjustments, establishing a multi-level correction strategy. This prioritization follows the following rules: first, correct significant deviations that could pose safety risks, such as knee hyperextension; second, correct critical deviations that impact movement effectiveness, such as insufficient hip angle; and finally, correct aesthetic deviations, such as an uneven arm arc. For example, during a squat, if the system detects a safety risk due to a low knee angle (65°) and an excessive forward center of gravity tilt, it prioritizes knee angle adjustment instructions, followed by center of gravity position adjustments.

[0179] The system converts the sorted adjustments into specific corrective movement instructions. For example, in yoga's Warrior pose, if the system detects insufficient external rotation of the right leg and excessive forward leaning, the system generates corrective instructions such as: "Rotate the right leg outward 15 degrees (adjust direction) with a moderate amplitude (adjust amplitude), complete within the next breathing cycle (execution timing)" and "Slow down the forward lean of the trunk by 40% (adjust direction and amplitude), execute immediately (execution timing)." These instructions can be delivered to the user through voice prompts, visual images, or tactile feedback.

[0180] Through the detailed steps above, the system can accurately compare user movements with standard movements and generate targeted adjustment instructions to help users effectively correct movement errors, improving exercise effectiveness and safety. This method is particularly suitable for scenarios that require precise movement guidance, such as fitness training, rehabilitation therapy, and dance learning.

[0181] The motion capture and feedback adjustment system for Metaverse Intelligent Movement according to an embodiment of the present invention includes:

[0182] The first unit is configured to collect actual motion data of the user during exercise through multiple inertial measurement units; establish a mapping relationship between the actual motion data and preset human skeleton nodes, and calculate the three-dimensional spatial position coordinates of each skeleton node based on the mapping relationship to obtain skeleton point position data;

[0183] The second unit is configured to construct a multi-dimensional scoring matrix based on the motion coherence index and the action standardization index, evaluate and score each skeletal node in the skeletal point position data in real time, and calculate the action score of each skeletal node; optimize the position of the skeletal nodes in the skeletal point position data whose scores are lower than a preset scoring threshold according to the action score, and generate optimized skeletal point position data;

[0184] A third unit is configured to construct a user virtual image using the optimized skeletal point position data, calculate virtual motion data of the user virtual image, and display the user virtual image in a metaverse virtual scene;

[0185] A fourth unit is configured to perform a time series analysis on the virtual motion data, calculate the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extract the complete motion sequence using a long-time analysis method to obtain global motion features;

[0186] The fifth unit is used to compare the local motion features and global motion features with the standard motion features, generate specific angle adjustment amounts, speed adjustment amounts and displacement adjustment amounts, convert the adjustment amounts into motion correction instructions, and guide the user to adjust the motion posture in real time.

[0187] According to a third aspect of an embodiment of the present invention, an electronic device is provided, including:

[0188] processor;

[0189] a memory for storing processor-executable instructions;

[0190] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0191] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0192] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0193] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A motion capture and feedback adjustment method for metaverse intelligent motion, characterized in that: include: The user's actual motion data during exercise is collected through multiple inertial measurement units; Establishing a mapping relationship between the actual motion data and preset human skeleton nodes, and calculating the three-dimensional spatial position coordinates of each skeleton node according to the mapping relationship to obtain skeleton point position data; A multi-dimensional scoring matrix is constructed based on the motion coherence index and the action standardization index, and each bone node in the bone point position data is evaluated and scored in real time to calculate the action score of each bone node; Optimizing the positions of the skeleton nodes in the skeleton point position data whose scores are lower than a preset score threshold according to the action score, and generating optimized skeleton point position data; Constructing a user virtual image using the optimized skeletal point position data, calculating virtual motion data of the user virtual image, and displaying the user virtual image in a metaverse virtual scene; Performing a time series analysis on the virtual motion data, calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extracting the complete motion sequence using a long-time analysis method to obtain global motion features; Comparing the local motion features and the global motion features with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts, and displacement adjustment amounts, and converting the adjustment amounts into motion correction instructions to guide the user to adjust their motion posture in real time; Sampling the skeletal point position data at fixed time intervals to generate a continuous motion data sequence, wherein the motion data sequence records the motion trajectory of each skeletal node over time; Segmenting the action data sequence into a plurality of overlapping action segments according to a preset time window length, wherein the overlapping rate of adjacent action segments is 50%; Calculating motion characteristic parameters of each skeletal node in each action segment, wherein the motion characteristic parameters include position change rate, velocity change rate, and acceleration change rate, and taking a weighted sum of the position change rate, velocity change rate, and acceleration change rate as the coherence score of the action segment; Retrieving a standard action sequence that matches the current action clip from a pre-labeled standard action library, wherein the standard action sequence includes a standard motion trajectory of each skeletal node under an ideal state; Comparing the actual motion trajectory of each skeletal node in the current action segment with the standard motion trajectory, calculating a trajectory deviation value, and calculating a standardization score of the action segment based on the trajectory deviation value; Mapping the coherence score and the standardization score to different dimensions of a multi-dimensional scoring matrix, respectively, setting dynamic weight coefficients for the different dimensions, and the weight coefficients are adaptively adjusted according to the action type; The scores of each dimension in the multi-dimensional scoring matrix are multiplied by the corresponding weight coefficients and summed up to obtain the action score of each skeletal node in the current action segment.

2. The method according to claim 1, characterized in that Establishing a mapping relationship between the actual motion data and the preset human skeleton nodes, and calculating the three-dimensional spatial position coordinates of each skeleton node according to the mapping relationship to obtain the skeleton point position data includes: The actual motion data includes acceleration data and angular velocity data obtained by a plurality of inertial measurement units; performing coordinate system alignment on the actual motion data, and converting the local coordinate systems of the plurality of inertial measurement units into a global coordinate system; Constructing a human skeleton chain structure, the human skeleton chain structure including trunk nodes, limb nodes and joint nodes, and presetting length constraints and angle constraints between each node; Calculating the initial positions and initial postures of the multiple inertial measurement units in the global coordinate system, and binding the initial positions and initial postures to corresponding nodes in the human skeleton chain structure; calculating the displacement change of each inertial measurement unit based on the acceleration data, and calculating the posture change of each inertial measurement unit based on the angular velocity data; According to the displacement change and the posture change, combined with the length constraint and the angle constraint, the three-dimensional spatial position coordinates of each node in the human skeleton chain structure are calculated.

3. The method according to claim 1, characterized in that Performing time series analysis on the virtual motion data and calculating the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features includes: The virtual motion data is sampled at a fixed frame rate, and N adjacent motion frames are grouped into a motion analysis window, where the number of overlapping frames between adjacent motion analysis windows is M; and the position changes of the skeletal nodes between adjacent motion frames within each motion analysis window are calculated, including spatial displacement vectors and angle change vectors. Based on the position change, a temporal correlation matrix is constructed between the action frames, wherein the matrix elements of the temporal correlation matrix represent the motion correlation of the corresponding skeletal nodes between adjacent frames; the temporal correlation matrix is decomposed in the time domain to extract the velocity features, acceleration features and angular velocity features between adjacent action frames; Calculate the motion consistency coefficient of each skeletal node in the motion analysis window, which is used to characterize the coherence of the local motion segment; identify abnormal motion frames in the motion analysis window based on the motion consistency coefficient, and mark motion frames with continuity feature values lower than a preset continuity threshold as abnormal frames; Based on the velocity features, acceleration features and angular velocity features, combined with the motion consistency coefficient, a feature vector representing the local motion features is generated; the feature vector of the local motion features is associated with the feature vector of the adjacent motion analysis window to construct a complete local motion feature sequence.

4. The method according to claim 1, wherein The global action features obtained by extracting the complete action sequence using the long-term analysis method include: Performing a time series analysis on the virtual motion data to obtain motion sequence data, the motion sequence data including three-dimensional coordinate information, velocity information, and acceleration information of each skeletal node from the start to the end of the motion; dividing the motion sequence data into a preparation phase, an acceleration phase, a stabilization phase, a deceleration phase, and an end phase in a temporal dimension according to a change trend of the motion state of the skeletal node; Feature extraction is performed on the motion data of each stage, and the average velocity, maximum acceleration, displacement amplitude and angle change range of the skeletal nodes in each stage are calculated to generate the stage feature vector; the motion state changes between adjacent stages are analyzed, and the velocity continuity and position smoothness of the skeletal nodes at the stage transition moment are calculated to generate the transition feature vector; The stage feature vector and the transformation feature vector are concatenated into a global action feature.

5. The method according to claim 4, characterized in that The method further comprises: Based on the stage feature vector and the conversion feature vector, a two-dimensional time series feature graph is constructed, where the horizontal axis represents the time dimension, the vertical axis represents the feature dimension, and the numerical value in the graph represents the feature intensity; Performing Fourier transform on the two-dimensional time series feature graph to obtain spectrum distribution, calculating the dynamic motion energy of each skeletal node, calculating the instantaneous motion energy of each frame based on the velocity vector and acceleration vector, and generating an energy time series curve; Aligning the energy time series curve with the two-dimensional time series feature graph in the time dimension, calculating the time series correlation between the feature intensity and the energy peak, and establishing a strength-feature mapping relationship; Based on the force-feature mapping relationship, the coordination degree between the force change and the motion feature conversion in the action sequence is analyzed to generate a coordination evaluation index.

6. The method according to claim 1, characterized in that Comparing the local motion features and the global motion features with the standard motion features to generate specific angle adjustment amounts, speed adjustment amounts, and displacement adjustment amounts, and converting the adjustment amounts into motion correction instructions includes: Retrieve standard action features that match the current action type from the action standard library, perform frame-level comparison between the local action features and the standard action features, calculate the posture deviation of each skeletal node between adjacent frames, and obtain real-time correction parameters; perform sequence-level comparison between the global action features and the standard action features, analyze the overall deviation trend during the action execution process, and obtain trend correction parameters; Based on the real-time correction parameters, the angle change amount that needs to be adjusted for each skeletal node is calculated, and the angle change amount is used to correct the degree of joint bending and the direction of rotation; based on the trend correction parameters, the speed adjustment amount of the skeletal node is calculated, and the speed adjustment amount is used to optimize the movement rhythm and movement smoothness; Calculate the displacement adjustment of the skeletal node in three-dimensional space by combining the angle change and the speed adjustment to ensure that the adjusted motion trajectory meets the standard requirements; Prioritize the angle change, speed adjustment, and displacement adjustment, establish a multi-level correction strategy, and avoid conflicts between multiple adjustment instructions; convert the sorted adjustment amounts into specific motion correction instructions, which include adjustment direction, adjustment amplitude, and execution timing.

7. A motion capture and feedback adjustment system for Metaverse Intelligent Movement, configured to implement the method according to any one of claims 1 to 6, characterized in that: include: The first unit is used to collect actual motion data of the user during exercise through multiple inertial measurement units; Establishing a mapping relationship between the actual motion data and preset human skeleton nodes, and calculating the three-dimensional spatial position coordinates of each skeleton node according to the mapping relationship to obtain skeleton point position data; The second unit is used to construct a multi-dimensional scoring matrix based on the motion coherence index and the action standardization index, perform real-time evaluation and scoring on each bone node in the bone point position data, and calculate the action score of each bone node; Optimizing the positions of the skeleton nodes in the skeleton point position data whose scores are lower than a preset score threshold according to the action score, and generating optimized skeleton point position data; A third unit is configured to construct a user virtual image using the optimized skeletal point position data, calculate virtual motion data of the user virtual image, and display the user virtual image in a metaverse virtual scene; A fourth unit is configured to perform a time series analysis on the virtual motion data, calculate the continuity features between adjacent motion frames using a short-time analysis method to obtain local motion features, and extract the complete motion sequence using a long-time analysis method to obtain global motion features; The fifth unit is used to compare the local motion features and global motion features with the standard motion features, generate specific angle adjustment amounts, speed adjustment amounts and displacement adjustment amounts, convert the adjustment amounts into motion correction instructions, and guide the user to adjust the motion posture in real time.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images