Motion capture optimization method, system, and storage medium based on state machine control

Abstract

The application relates to the technical field of motion capture, and discloses a motion capture optimization method and system based on state machine control and a storage medium. The method comprises the following steps: mapping posture data to a foot static state; setting a state machine, and performing free integration on a root node; calculating a motion state and a foot angle, accumulating a state duration, and performing state detection based on the foot angle and the state duration if the static state; updating the state duration if the state does not change, and detecting whether a height change occurs on a contact plane; updating a reference height if the height changes; selecting an inverse kinematics method correction scheme based on the state machine: if single-foot landing occurs, correcting the height of the single foot to the reference height, and then performing position correction; if double-foot landing occurs, performing position correction by using a whole-body inverse kinematics method; and if double-foot landing does not occur, performing free integration on the root node, and updating a whole-body node, so that the accuracy of motion position restoration is improved in a motion capture process.

Description

Technical Field

[0001] This application relates to the technical field of motion capture, and in particular to a motion capture optimization method, system, and storage medium based on state machine control. Background Technology

[0002] Current motion capture technology is mainly divided into two categories: optical and inertial. Optical motion capture uses multiple high-speed cameras to capture reflected light from marker points and calculates the three-dimensional coordinates of the marker points to reconstruct the motion. It has the advantage of high precision, but it is easily affected by occlusion and has high requirements for the site. Inertial motion capture relies on inertial measurement units (IMUs) to collect angular velocity and acceleration data, and combines them with algorithms to solve the motion posture. It does not require external optical equipment, is not limited by occlusion, and can realize motion capture in moving scenes. Both have wide applications in film and television production, game development, motion analysis, and other fields.

[0003] The rapid development of microelectromechanical systems (MEMS) technology has driven the miniaturization, low power consumption, and low cost upgrades of inertial measurement units (IMUs). MEMS-based IMUs can be directly integrated into motion capture devices to collect inertial parameters of human motion in real time without relying on external infrastructure.

[0004] In existing motion capture technologies, some solutions only design stationary detectors for the feet, using the determination of whether the feet are stationary to stabilize the root node. However, this method is susceptible to external disturbances, such as minor bumps in the ground during walking, unconscious slight swaying of the feet, or friction from clothing and environmental vibrations during movement. These can all cause the foot stationary detector to misjudge, causing the foot to jump out of the preset stationary state. Once the root node loses its stationary reference, it will produce unnecessary positional movement. Summary of the Invention

[0005] To improve the accuracy of motion position reconstruction during motion capture, this application provides a motion capture optimization method, system, and storage medium based on state machine control.

[0006] Firstly, this application provides a motion capture optimization method based on state machine control, employing the following technical solution:

[0007] A motion capture optimization method based on state machine control includes the following steps:

[0008] S1: Initialize the human body model based on the set skeletal framework, and acquire posture quaternions, accelerometer data and gyroscope data from the acquisition device;

[0009] S2: Calculate posture data based on generalized likelihood ratio test and map the posture data to the foot static state;

[0010] S3: Set and initialize the state machines for the left and right feet. The states of the state machines for the left and right feet include fully landed, foot landed and toes raised, foot landed and toes landed, heel raised, heel landed and swaying. The initial state of the state machine is fully landed.

[0011] S4: Perform free integration on the root node centered on the pelvis based on attitude quaternions, accelerometer data, and gyroscope data;

[0012] S5: Calculate the motion state and foot angle based on accelerometer and gyroscope data, and accumulate the duration of the motion state. If the motion state is stationary, perform state detection based on the foot angle and state duration. If the state does not change, update the state duration and detect whether the contact plane has changed in height based on the slope detector. If the contact plane has changed in height, update the reference height corresponding to the contact plane based on the calculation data of the slope detector.

[0013] S6: Correction scheme based on the inverse kinematics method corresponding to the state selection of the state machine: If one foot is on the ground, the height of the foot is corrected to the reference height, and the position of the human skeletal chain from the foot to the root node is corrected; if both feet are on the ground, the position is corrected using the whole-body inverse kinematics method; if neither foot is on the ground, after the state machine is in the state of toe raised or heel raised, the root node is free integrated, and then the whole-body nodes are updated using the forward kinematics method.

[0014] By adopting the above technical solutions, state machines offer a robust control method without relying on optical, UWB, or soft magnetic systems. This is suitable for motion capture gait with clearly defined states, such as toe-to-toe contact, full contact, heel-to-toe contact, and full lift. Considering the IMU's accuracy, installation and calibration conditions, and execution limitations, the foot may still experience clipping and slippage upon contact; therefore, a reference plane is used for constraint. Traditional inverse dynamics and its variants, such as Jacobin-IK and CCD-IK, have many problems with correcting multi-chain leg systems, such as high computational cost and non-unique convergence solutions. Using dual-bone IK methods, such as Two-Bone-IK and TBIK, is superior in terms of both computational cost and unique convergence solution. This can improve the performance of motion capture gait upon landing and, combined with adaptive static detectors, slope detectors, and other state machine conditions, achieve compatibility with highly dynamic dimensions and uneven terrain, thereby improving the accuracy of motion position reconstruction during motion capture.

[0015] Optionally, step S1 may further include the following sub-steps:

[0016] Set the node structure as: Joint{quat, acc, omg, static_status, lever};

[0017] Where quat represents the attitude in the global coordinate system, acc represents accelerometer data, omg represents gyroscope data, static_status represents the stationary state, and lever represents the length of the bone with this node as its parent node; the parent node of a bone is the starting point of its segment, i.e., recursively updating the nodes of the human body from the pelvis as the root node towards the end nodes of the body. Based on forward dynamics, the nodes of the human body are recursively updated as follows:

[0018] ;

[0019] Where R(quat) is a function that transforms a quaternion into a direction cosine matrix. and It is a three-dimensional position vector. This is a three-dimensional lever arm vector.

[0020] By adopting the above technical solutions, the node structure and forward dynamics recursion can integrate multi-dimensional posture data, simplify the skeletal chain position update logic, reduce computational redundancy, and improve the coherence and accuracy of posture recursion of each node in the human body.

[0021] Optionally, step S2 may further include the following sub-steps:

[0022] Set the input sequence as follows: ;

[0023] in, Given the input data sequence, and Let be the accelerometer and gyroscope data at time k, respectively, and N be the window length; let be... At rest, For motion, a joint detection quantity is constructed. have:

[0024] ;

[0025] in, for ;

[0026] Set detection threshold ,when When, the state is set to Set static_status to true; otherwise, set the status to true. Set static_status to false;

[0027] If the number of data inputs is not the same as the number of skeletal nodes, then the static state of the skeletal nodes is mapped, that is, one data input corresponds to the static state of multiple nodes.

[0028] By adopting the above technical solution, setting the input sequence and constructing a joint detection quantity, and combining the detection threshold judgment, the static and dynamic states of skeletal nodes can be accurately distinguished, improving the reliability of static detection; the state mapping design when the number of data and nodes is different is conducive to adapting to data input in multiple scenarios.

[0029] Optionally, step S3 may further include the following sub-steps:

[0030] Initialize the state machines for the left and right feet. The states of each state machine include fully landed (FF), foot landed and toes raised (TC), foot landed and toes landed (TF), heel raised (HC), heel landed (HF), and swaying (SW).

[0031] The state change quantity is calculated within a preset detection time. The detection threshold is adjusted according to the inverse correlation of the state change quantity. The larger the state change quantity, the smaller the detection threshold, and vice versa.

[0032] By adopting the above technical solution, different landing and movement postures of the foot can be accurately identified; the detection threshold can be adjusted inversely according to the change in state, which can dynamically adapt to state fluctuations and improve the sensitivity and accuracy of state judgment.

[0033] Optionally, step S4 may further include the following sub-steps:

[0034] Based on attitude quaternions, accelerometer data, and gyroscope data, perform a free integral on the root node, as shown in the following formula:

[0035] ;

[0036] ;

[0037] in, This is the value of gravitational acceleration. The time for integration.

[0038] By adopting the above technical solution and integrating multi-dimensional data to perform free integration on the root node, the position and velocity of the root node can be updated in real time, which helps to reduce the impact of errors from single data.

[0039] Optionally, step S5 may further include the following sub-steps:

[0040] When `static_status` is true for more than the preset number of reference frames consecutively, the state machine should be reset to a fully committed state. The state transition condition is defined as follows:

[0041] C1: The absolute value of the difference in accelerometer modulus between frame k-1 and frame k is greater than the set first threshold;

[0042] C2: The absolute value of the difference between the accelerometer modulus and the local gravitational acceleration value is greater than the set second threshold;

[0043] C3: The angular velocity modulus is greater than the set third threshold;

[0044] C4: The pitch angle angular velocity is less than or greater than the set fourth threshold;

[0045] C5: Pitch angle value is greater than or less than 0;

[0046] C6: not-static_status=true;

[0047] C7: The duration of the state exceeds the set minimum time;

[0048] Then, for different jump states, we have:

[0049]

[0050] Where * indicates that the condition only needs to be satisfied once;

[0051] If no state transition is detected in the current frame, the duration of the current state is incremented.

[0052] A slope detector is used to calculate slope change indicators. The slope detector acquires the components of acceleration data in the sensor coordinate system to calculate the slope angle. In the sensor coordinate system, the x-axis is the forward direction, and the z-axis is perpendicular to the upper surface. In a stationary state, the gravitational acceleration g only produces a component along the z-axis. When on an inclined slope, the gravitational acceleration is decomposed into components along the x-axis and z-axis, forming component a. x The formula for calculating the slope angle α is: α = arctan(|a_z|). x | / a_z); where arctan is the arctangent function;

[0053] The reference height is calculated based on the slope angle α and the preset length data.

[0054] By adopting the above technical solutions, continuous static reset can ensure the stability of the foot reference, and multiple jump conditions are conducive to the accurate identification of movement changes; slope detection combined with reference height calculation can adapt to sloping terrain, provide a realistic height reference for skeletal correction, and help improve the accuracy of motion capture in complex terrain.

[0055] Optionally, step S6 may further include the following sub-steps:

[0056] If the landing is on one foot, the percentage of the first occurrence of landing on one foot is calculated. If the percentage of the first occurrence is greater than the preset reference occurrence percentage, window filtering is applied to multiple reference heights. The ratio between the percentage of the first occurrence and the preset reference occurrence percentage is calculated as the first ratio. The window length of the reference height is adjusted according to the positive correlation of the first ratio.

[0057] If neither foot is on the ground, calculate the percentage of states where neither foot is on the ground. If the percentage of states is within the preset first percentage range, perform free integration on the root node after the toes are lifted; otherwise, perform free integration on the root node after the heels are lifted.

[0058] By adopting the above technical solution, when one foot is on the ground, the decision to perform filtering is made based on the proportion, and the window length is adjusted according to the proportion, which helps to improve the stability of the reference height; when both feet are not on the ground, the integration timing is adapted according to the proportion range, which helps to reduce the root node integration deviation.

[0059] Optionally, the ratio between the changed state quantity and the preset reference state quantity is calculated as the state ratio.

[0060] The bias value of the first proportion range is adjusted according to the positive correlation of the state ratio. The larger the state ratio, the larger the bias value, and the smaller the state ratio, the smaller the bias value.

[0061] The number of reference frames is adjusted inversely based on the state ratio. The larger the state ratio, the smaller the number of reference frames, and vice versa.

[0062] By adopting the above technical solution, the bias value of the first proportion range can be adjusted according to the state ratio to adapt to different state fluctuation levels; the reference frame number can be adjusted according to the inverse correlation of the state ratio to balance the sensitivity and stability of state judgment.

[0063] Secondly, this application provides a motion capture optimization system based on state machine control, which adopts the following technical solution:

[0064] A motion capture optimization system based on state machine control includes a processor, wherein the processor executes the steps of the motion capture optimization method based on state machine control as described in any of the preceding claims.

[0065] Thirdly, this application provides a storage medium, which adopts the following technical solution:

[0066] A storage medium storing a program, which, when executed by a processor, implements the steps of the motion capture optimization method based on state machine control described above.

[0067] In summary, this application includes at least one of the following beneficial technical effects: State machines offer a robust control method without relying on optical, UWB, or soft magnetic systems, suitable for motion capture gait with clearly defined states, such as toe-to-toe contact, full contact, heel-to-toe contact, and full lift-off. Considering the accuracy, installation calibration conditions, and execution limitations of the IMU, the foot may still experience clipping and slippage upon contact; therefore, a reference plane is chosen for constraint. Traditional inverse dynamics and its variants, such as Jacobin-IK and CCD-IK, have many problems with the correction of multi-chain leg systems, such as high computational cost and non-unique convergence solutions. Using bi-bone IK methods, such as Two-Bone-IK and TBIK, is a superior approach in terms of both computational cost and unique convergence solution. This can improve the performance of motion capture gait upon landing and, in conjunction with adaptive static detectors, slope detectors, and other state machine conditions, achieve compatibility with highly dynamic dimensions and uneven terrain, thereby improving the accuracy of motion position reconstruction during motion capture. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating the motion capture optimization method based on state machine control. Detailed Implementation

[0069] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0070] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0071] This application discloses a motion capture optimization method based on state machine control, referring to... Figure 1 It includes the following steps:

[0072] S1: Human model initialization and data acquisition.

[0073] The human model is initialized based on a predefined skeletal framework. This framework is built recursively towards the distal nodes of the body, with the pelvis as the root node, following the natural connections of the human physiological structure. For example, pelvis → left hip → left knee → left ankle → left toe, and the same applies to the right skeletal chain. The structure of each skeletal node is defined as: Joint{quat, acc, omg, static_status, lever}.

[0074] The specific meanings and functions of each parameter are as follows:

[0075] quat: Represents the pose quaternion in the global coordinate system, used to accurately describe the orientation information of bone nodes in three-dimensional space;

[0076] acc: Represents the accelerometer data output by the acquisition device (such as MEMS inertial measurement unit (IMU)), with the unit being m / s², used to reflect the acceleration changes at the node;

[0077] omg: This is the gyroscope data output by the acquisition device, measured in rad / s, used to represent the change in the angular velocity of the node;

[0078] static_status: This is a boolean value that indicates the node is in a static state. When static_status is true, it means the node is in a static state, and when it is false, it means the node is in a moving state.

[0079] lever: refers to the bone length vector with the current node as the parent node. It is a three-dimensional vector with units of meters and is used to determine the position offset of the child node relative to the parent node.

[0080] Based on the principle of forward dynamics, the position of a child node is derived from the position and orientation of its parent node, and the calculation formula is as follows:

[0081] ;

[0082] R(quat) is a function that converts quaternions into a direction cosine matrix. This function can convert the attitude information represented by quaternions into the form of a direction cosine matrix for subsequent vector transformations and other operations. and These are the position vectors of the child node and the parent node in three-dimensional space, respectively, used to specify the spatial position of the node; It is a 3D lever vector that represents the offset of the child node position relative to the parent node. It combines the parent node's pose information to determine the final position of the child node.

[0083] This node structure definition and forward dynamics recursion approach integrates multi-dimensional posture data, such as quaternions, acceleration, and angular velocity, effectively simplifying the logic of skeletal chain position updates and avoiding complex and redundant calculations. Furthermore, this recursive approach follows the natural connections and movement patterns of the human skeleton, significantly improving the coherence and accuracy of posture recursion across different nodes, providing precise foundational data support for subsequent state machine-based motion capture optimization.

[0084] S2: Calculate posture data based on generalized likelihood ratio test and map it to the foot's stationary state.

[0085] By using the generalized likelihood ratio test (GLRT), the static or dynamic state of foot skeletal nodes is accurately determined, and a reasonable mapping of states is achieved when the data input does not match the number of skeletal nodes, providing key state basis for subsequent state machine-based motion capture optimization.

[0086] Set the input sequence as follows: ;

[0087] in, The input data sequence is used to carry attitude-related data over a period of time; and These are the accelerometer and gyroscope data at time k, respectively, which are the core data sources reflecting the foot's movement state; N is the window length. The selection of the window length needs to take into account both computational efficiency and detection accuracy. The recommended value range is 50-100 frames, which can be flexibly adjusted according to the sampling rate of the actual acquisition device to adapt to different application scenarios.

[0088] Construction of the hypothesis testing procedure. Let... With the feet stationary, the acceleration is approximately equal to the acceleration due to gravity, and the angular velocity is approximately zero. Since the foot is in motion, its acceleration and angular velocity will deviate from the characteristics of the stationary state. Therefore, a joint detection quantity is constructed. have:

[0089] ;

[0090] in, g is the local gravitational acceleration, approximately 9.8 m / s², and this vector is used to characterize the baseline case of acceleration in a stationary state; and The deviations of gyroscope and accelerometer data from a stationary baseline were calculated separately. These deviations were then averaged within a window to obtain a joint detection value T, which comprehensively reflects the foot's motion characteristics.

[0091] Set detection threshold The recommended value range is 0.1-0.5. This threshold can be calibrated through extensive experimentation to adapt to different acquisition devices and application environments. At that time, the foot condition was determined to be consistent with the null hypothesis. In this case, set the node's static_status to true; otherwise, determine that the foot state meets the alternative hypothesis. In other words, when the foot is in motion, set static_status to false.

[0092] Considering the possibility of a mismatch between the number of data inputs and the number of skeletal nodes in practical applications (for example, a single IMU device may need to correspond to multiple foot skeletal nodes), it is necessary to map the resting states of the skeletal nodes. That is, to map the resting state determined by a single data input to all the relevant skeletal nodes it covers, ensuring that each skeletal node can obtain an accurate resting state identifier.

[0093] By setting input sequences, constructing joint detection metrics, and combining these with detection thresholds, the method can accurately distinguish between the static and dynamic states of skeletal nodes, effectively improving the reliability of static detection and avoiding deviations in subsequent motion capture optimization due to misjudgments of state. Furthermore, the state mapping design for different numbers of data points and nodes allows the method to flexibly adapt to data input conditions in various scenarios, enhancing its versatility and practicality.

[0094] S3: Setting and initializing the left and right foot state machines.

[0095] This step involves constructing independent state machines for the left and right feet respectively, completing the initialization operation, and establishing a dynamic detection threshold adjustment mechanism to accurately identify different landing and movement postures of the feet, providing a state foundation for subsequent motion capture optimization based on state machines.

[0096] Separate state machines are defined for the left and right feet. Each state machine contains the following six states, which cover the typical postures that the foot may take during movement:

[0097] Fully grounded (FF): This indicates that the entire foot remains in contact with the ground.

[0098] Heel-to-toe (TC): This means that the heel touches the ground while the toes are raised.

[0099] Heel-to-toe landing (TF): This refers to the state where the heel leaves the ground and the toes touch the ground.

[0100] Heel lift (HC): refers to the state where the heel is off the ground and the toes are touching the ground;

[0101] Heel strike (HF): This means the heel touches the ground and the toes are raised.

[0102] Swinging state (SW): Indicates that the feet are completely off the ground and in the process of swinging.

[0103] When initializing the state machine, the initial state of both the left and right foot state machines is set to fully grounded (FF). This is because when the human body is standing naturally or in a relatively stable initial motion state, the feet are usually in full contact with the ground. Using this as the initial state conforms to the general law of human movement and makes it easier for the state machine to make state transitions according to the actual movement situation.

[0104] The dynamic detection threshold is adjusted. Within a preset detection time, which can be set according to the actual application scenario and movement speed, for example, 0.1-0.5 seconds, the changes in foot state are calculated. These changes can be obtained by statistically analyzing the changes in foot acceleration, angular velocity, and other data over a period of time, reflecting the degree of fluctuation in foot state. Then, the detection threshold is adjusted based on the inverse correlation of the changes in state: the larger the changes in state, the more drastic the fluctuations in foot state, and the smaller the detection threshold should be to more sensitively capture these changes; conversely, the smaller the changes in state, the more stable the foot state, and the larger the detection threshold should be. This dynamic adjustment method allows the state machine to better adapt to changes in foot posture under different movement conditions.

[0105] By setting state machines for each foot, including various typical postures, and using a fully landed state that conforms to the laws of human movement as the initial state, different landing and movement postures of the feet can be accurately identified. Furthermore, the mechanism of adjusting the detection threshold based on the inverse correlation of state changes can dynamically adapt to state fluctuations, avoiding the problems of lag or misjudgment caused by fixed thresholds. This improves the sensitivity and accuracy of state judgment, providing a reliable state basis for subsequent state machine-based motion capture optimization.

[0106] S4: Perform free integration on the root node based on multi-dimensional data.

[0107] The purpose of this step is to fuse attitude quaternions, accelerometer data, and gyroscope data to perform free integration on the root node centered on the pelvis, thereby updating the position and velocity of the root node in real time and providing accurate root node motion information for subsequent state machine-based motion capture optimization.

[0108] The multi-dimensional data sources used for integration are clearly defined. Attitude quaternions are used to accurately describe the spatial orientation of the root node; accelerometer data reflects the acceleration changes of the root node in various directions; and gyroscope data reflects the angular velocity of the root node. All of these data are acquired synchronously by acquisition devices (such as inertial measurement units (IMUs)), providing a comprehensive information foundation for the motion analysis of the root node.

[0109] Free integration is performed on the root node based on this multi-dimensional data. The specific integration formula is as follows:

[0110] ;

[0111] ;

[0112] The meanings and functions of each parameter are as follows:

[0113] vk and vk-1: These are the velocity vectors of the root node at time k and time k-1, respectively, used to characterize the speed and direction of the root node's motion at different times;

[0114] R(quat): is a function that converts attitude quaternions into direction cosine matrices. This function can convert the attitude information represented by quaternions into the form of direction cosine matrices, so as to transform the accelerometer data from the sensor coordinate system to the global coordinate system, thereby accurately calculating the acceleration influence of the root node in the global coordinate system.

[0115] ak: is the accelerometer data vector acquired at time k;

[0116] g: is the gravitational acceleration value, approximately 9.8 m / s², used to subtract the effect of gravity on acceleration during the integration process to obtain the actual motion acceleration of the root node;

[0117] dt: is the integration time, which is the time interval between two data acquisitions. It is determined by the sampling rate of the acquisition device. For example, when the sampling rate is 100Hz, dt=0.01s.

[0118] pk and pk-1: These are the position vectors of the root node at time k and time k-1, respectively, used to determine the specific location of the root node in three-dimensional space.

[0119] By integrating attitude quaternions, accelerometer data, and gyroscope data freely onto the root node, the complementary nature of multi-dimensional data can be fully utilized to update the root node's position and velocity in real time. This approach avoids the error accumulation problem caused by relying solely on single data sources (such as accelerometer data or gyroscope data alone) for integration, reduces the impact of errors from single data sources on the calculation of root node motion information, and provides more accurate and reliable root node motion foundation data for subsequent state machine-based motion capture optimization.

[0120] S5: Motion state and foot angle calculation and state machine management.

[0121] This step processes accelerometer and gyroscope data to calculate motion state and foot angle, combines state duration for state detection and machine jump control, and utilizes a slope detector to adapt to sloping terrain, providing accurate state and height references for motion capture.

[0122] First, the motion state and foot angle are calculated based on accelerometer and gyroscope data, and the duration of the motion state is accumulated. If the motion state is stationary, the state detection is performed based on the foot angle and the duration of the state to determine the posture characteristics of the foot when stationary.

[0123] If the state remains unchanged, the state duration is updated. Simultaneously, a slope detector checks for changes in the height of the contact surface. The slope detector works by acquiring the components of acceleration data in the sensor coordinate system to calculate the slope angle. In the sensor coordinate system, the x-axis represents the forward direction, and the z-axis is perpendicular to the upper surface of the equipment. In a stationary state, gravitational acceleration g only has a component along the z-axis; when on an inclined slope, gravitational acceleration is decomposed into components along both the x and z axes. and The formula for calculating the slope angle α is:

[0124] .

[0125] Here, arctan is the arctangent function. If a change in height is detected on the contact plane, a reference height is calculated and updated based on the slope detector's calculation data, namely the slope angle α and preset length data (such as bone-related lengths), so that subsequent bone correction can adapt to the sloping terrain.

[0126] Regarding slope detection, in addition to slopes, scenarios with stairs must also be considered (stair landing points are on a plane). In this case, the metric up_flag = |Δpitch(deg)*accel_vert(m / s^2)| needs to be calculated, with a threshold set to 200~300. When the foot switches from a fully grounded (FF) state, this metric will be continuously monitored. Once the threshold is met once, up_flag will be set to true (left and right feet are monitored separately). Subsequently, when the foot touches the ground (and is not in a swaying (SW) state), the difference between the current contact point height and the reference height is compared: if the difference is less than 4cm, the reference height is not updated; if the difference is greater than or equal to 4cm, the reference height is updated based on the current contact point height, and the skeletal height is corrected accordingly. After correction, up_flag is set to false.

[0127] When static_status is true for more than the preset number of reference frames, the state machine should be reset to the fully grounded state to ensure the stability of the foot reference.

[0128] The transition conditions for a state are defined by the following seven basic conditions:

[0129] C1: The absolute value of the difference between the accelerometer modulus of frame k-1 and frame k is greater than the set first threshold (the first threshold can be set according to the actual scene and the accuracy of the equipment, for example, 0.5m / s²).

[0130] C2: The absolute value of the difference between the accelerometer modulus and the local gravitational acceleration value is greater than the set second threshold (the second threshold is recommended to be 1.0 m / s²).

[0131] C3: Angular velocity modulus is greater than the set third threshold (the third threshold is recommended to be 1.0 rad / s).

[0132] C4: The pitch angle angular velocity is less than or greater than the set fourth threshold (e.g., in the foot-to-toe (TF) landing state, the corresponding threshold TFrate is recommended to be -1 rad / s).

[0133] C5: Pitch angle value is greater than or less than 0 (a pitch angle greater than 0 indicates toes pointing upwards, and a pitch angle less than 0 indicates toes pointing downwards);

[0134] C6: not-static_status is true (meaning the foot is in motion);

[0135] C7: The duration of the state exceeds the set minimum time (the recommended minimum time is 0.1-0.3s).

[0136] Based on the above jump conditions, the following specific rules apply to different jump states (where * indicates that the condition only needs to be met once):

[0137] This indicates the condition for transitioning from a fully landed (FF) state to a foot-landed (TF) state;

[0138] That is, the condition for jumping from the full landing (FF) state to the heel landing (HF) state;

[0139] This is the condition for transitioning from the foot-to-toe (TF) state to the swing (SW) state;

[0140] This is the condition for transitioning from the heel-to-slip (HF) state to the swing (SW) state;

[0141] ,here The acceleration is in the vertical direction, which is the condition for jumping from the swaying state (SW) to the foot-landing and toe-lifting state (TC);

[0142] That is, the condition for jumping from the swaying state (SW) to the heel lift (HC) state;

[0143] This is the condition for transitioning from the foot-on-the-foot-lift (TC) state to the swinging (SW) state.

[0144] If no state transition is detected in the current frame, the duration of the current state is incremented.

[0145] The continuous static reset mechanism ensures foot baseline stability and avoids state shifts caused by prolonged stillness. Multiple jump conditions combine acceleration, angular velocity, angle, and duration information to accurately identify motion changes. Furthermore, slope detection combined with reference height calculation adapts to sloping terrain, providing a realistic height baseline for subsequent skeletal correction, thus improving the accuracy of motion capture in complex terrain.

[0146] S6: A modified scheme for the inverse kinematics method based on the state selection of the state machine.

[0147] This step selects the corresponding inverse kinematics (IK) method to correct the bone position based on the different states of the state machine, in order to adapt to different foot landing conditions and terrain, and improve the accuracy of motion capture.

[0148] Without relying on external systems such as optics, UWB (ultra-wideband), and soft magnets, state machines offer a robust control approach, making them ideal for motion capture gait scenarios with clearly defined states (e.g., toe-to-toe, full-to-toe, heel-to-toe, and full-to-lift). However, considering the accuracy limitations, installation and calibration conditions, and execution requirements of the IMU (Inertial Measurement Unit), foot slippage and clipping can still occur during ground contact. Therefore, a reference plane is chosen to constrain the foot's position.

[0149] Traditional inverse dynamics and its variants, such as Jacobin-IK and CCD-IK, suffer from numerous problems when modifying multi-chain leg systems, including high computational cost and non-unique convergent solutions. However, methods employing dual-bone IK (such as Two-Bone-IK and TBIK) perform IK calculations through dual chains, exhibiting superior performance in terms of both computational cost and unique convergent solutions. This approach improves performance during gait landing in motion capture and can be combined with adaptive stationary detectors, slope detectors, and other state machine conditions to achieve compatibility with highly dynamic dimensions and uneven terrain, thereby enhancing the accuracy of motion position reconstruction during motion capture.

[0150] The specific revision plan is as follows:

[0151] Single-leg landing scenario: If landing on one leg, first correct the height of that leg to the reference height, then correct the position of the human skeletal chain from that leg to the root node. Calculate the percentage of first-occurrence single-leg landings. If this percentage is greater than the preset reference percentage, it indicates that the reference height may fluctuate significantly. In this case, window filtering is applied to multiple reference heights to smooth the reference height. The ratio between the first-occurrence percentage and the preset reference percentage is calculated as the first ratio. The window length of the reference height is adjusted based on a positive correlation with the first ratio; that is, the larger the first ratio, the longer the window length to more effectively filter out fluctuations; the smaller the first ratio, the shorter the window length to avoid over-filtering and response delay.

[0152] When both feet land: If both feet land, use the whole-body inverse kinematics method to correct the position and ensure that the position and posture of the whole body skeleton when both feet land are consistent with the actual movement.

[0153] Case where neither foot is on the ground: If neither foot is on the ground, after the state machine is in the state of toe raised or heel raised, perform free integration on the root node, and then update the whole-body nodes using a forward dynamics method. Calculate the percentage of states where neither foot is on the ground. If the percentage of such states is within a preset first percentage range, then perform free integration on the root node after the toe is raised; otherwise, perform free integration on the root node after the heel is raised.

[0154] Furthermore, the ratio between the changing state quantity and the preset reference state quantity is calculated as the state ratio. The bias value of the first proportion range is adjusted based on a positive correlation with the state ratio. A larger state ratio indicates more drastic state fluctuations, requiring a larger bias value to expand the first proportion range and accommodate more complex state changes; a smaller state ratio results in a smaller bias value, narrowing the first proportion range and ensuring the accuracy of integration timing. Simultaneously, the number of reference frames is adjusted based on an inverse correlation with the state ratio. A larger state ratio results in a smaller number of reference frames, improving the sensitivity of state judgment; a smaller state ratio results in a larger number of reference frames, enhancing the stability of state judgment, thus balancing the sensitivity and stability of state judgment.

[0155] When landing on one foot, determining whether to perform filtering and adjusting the window length based on the percentage of the landing height helps improve the stability of the reference height and avoids deviations in skeletal correction due to instability in the reference height. When neither foot is on the ground, adapting the integration timing according to the percentage range helps reduce the root node integration deviation and makes the motion information of the root node more accurate. Adjusting the first percentage range offset value according to the state ratio can adapt to different degrees of state fluctuation; adjusting the reference frame number according to the inverse correlation of the state ratio can achieve a balance between the sensitivity and stability of state judgment.

[0156] This application also discloses a motion capture optimization system based on state machine control, including a processor, wherein the processor executes the steps of the motion capture optimization method based on state machine control as described in any of the above embodiments.

[0157] This application also discloses a storage medium storing a program that, when executed by a processor, implements the steps of the motion capture optimization method based on state machine control described above.

[0158] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A motion capture optimization method based on state machine control, characterized in that, The method comprises the following steps: S1: initializing a human body model based on a set bone framework, and acquiring posture quaternions, accelerometer data and gyroscope data from a collection device; S2: calculating posture data based on a generalized likelihood ratio test, and mapping the posture data to a foot static state; S3: setting and initializing state machines of left and right feet, and the states of the state machines of the left and right feet include full landing, foot landing toe lifting, foot landing toe landing, heel lifting, heel landing and swinging, wherein the initial state of the state machine is full landing; S4: performing free integration on a root node with a pelvic bone as a center based on the posture quaternions, the accelerometer data and the gyroscope data; S5: calculating a motion state and a foot angle according to the accelerometer data and the gyroscope data, accumulating a state duration of the motion state, and performing state detection based on the foot angle and the state duration if the motion state is a static state; if the state does not change, updating the state duration, and detecting whether a height of a contact plane changes based on a slope detector; if the height of the contact plane changes, updating a reference height corresponding to the contact plane according to calculation data of the slope detector; S6: selecting a corresponding inverse kinematics method correction scheme based on the state of the state machine: if one foot lands, correcting a height of the foot to the reference height, and then performing position correction on a human body bone chain from the foot to the root node; if both feet land, performing position correction by using a whole body inverse kinematics method; if neither foot lands, performing free integration on the root node after the state machine is in a toe lifting state or a heel lifting state, and then updating all body nodes by using a forward dynamics method.

2. The state machine control based motion capture optimization method of claim 1, wherein, In step S1, the following sub-steps are further included: The node structure is set as: Joint{quat, acc, omg, static_status, lever}; Wherein, quat is a posture in a global coordinate system, acc is accelerometer data, omg is gyroscope data, static_status is a static state, and lever is a bone length of a node serving as a parent node; a parent node of a bone is a starting point of a segment of the bone, that is, the pelvic bone is recursively updated to a terminal node of the human body in a direction, and the nodes of the human body are recursively updated based on forward dynamics as follows: ； where R(quat) is a function that converts quaternions to direction cosine arrays, and is a three-dimensional position vector, is a three-dimensional link vector.

3. The state machine control based motion capture optimization method of claim 2, wherein, In step S2, the following sub-steps are further included: The input sequence is set as: ; wherein, is the input data sequence, and are the accelerometer and gyroscope data at time k, respectively, and N is the window length; let be stationary, be moving, then construct the joint detection quantity has: ； wherein is a vector; Setting a detection threshold When the status is set to static_status is set to true, otherwise the status is set to static_status is set to false; If the number of data inputs is different from the number of bone nodes, the static states of the bone nodes are mapped, that is, one data input corresponds to the static states of multiple nodes.

4. The state machine control based motion capture optimization method of claim 3, wherein, In step S3, the following sub-steps are further included: The state machines of the left and right feet are initialized, and the states of each state machine include full landing FF, foot landing toe lifting TC, foot landing toe landing TF, heel lifting HC, heel landing HF and swinging SW; The change state quantity of the state is calculated within a preset detection time, and the detection threshold value is inversely correlated adjusted according to the change state quantity, that is, the greater the change state quantity, the smaller the detection threshold value, and the smaller the change state quantity, the greater the detection threshold value.

5. The state machine control based motion capture optimization method of claim 4, wherein, In step S4, the following sub-steps are further included: Based on the attitude quaternion, accelerometer data and gyroscope data, the root node is integrated once freely, and the formula is as follows: ； ； wherein is the value of the gravitational acceleration, is the integration time.

6. The state machine control based motion capture optimization method of claim 5, wherein, The step S5 further includes the following sub-steps: When static_status is true continuously for more than a preset reference frame number, the state of the state machine should be reset to fully land, and the state transition condition is defined as: C1: the absolute value of the difference between the k-1 frame and the k frame accelerometer module length is greater than the set first threshold value; C2: the absolute value of the difference between the accelerometer module length and the local gravity acceleration value is greater than the set second threshold value; C3: the angular velocity module length is greater than the set third threshold value; C4: the pitch angle angular velocity is less than or greater than the set fourth threshold value; C5: the pitch angle value is greater than or less than 0; C6: not-static_status=true; C7: the state duration exceeds the set minimum time; Then for different transition states: Wherein, * indicates that only 1 time needs to be met; If no state transition is found in the current frame, the duration in the current state is accumulated; And using the slope detector to calculate the slope change mark, the slope detector obtains the component of the acceleration data in the sensor coordinate system to calculate the slope angle; in the sensor coordinate system, the x axis is the forward direction, the z axis is perpendicular to the upper surface, and in the static state, the gravity acceleration g only generates a component in the z axis; when being on the inclined slope, the gravity acceleration will be decomposed into the x axis and the z axis to form the components a x and a_z; the calculation formula of the slope angle a is: a = arctan(|a x | / a_z); wherein, arctan is the inverse tangent function; According to the slope angle α and the preset length data, the reference height is calculated.

7. The state machine control based motion capture optimization method of claim 6, wherein, The step S6 further includes the following sub-steps: If single-foot landing, calculate the first appearance ratio of single-foot landing, if the first appearance ratio is greater than the preset reference appearance ratio, then window filter the plurality of reference heights; Calculate the ratio between the first appearance ratio and the preset reference appearance ratio as the first ratio, and adjust the window length of the reference height according to the positive correlation of the first ratio; If both feet are not landed, calculate the appearance state ratio of both feet not landing, if the appearance state ratio is within the preset first ratio range, then integrate the root node freely after the foot tip is lifted, otherwise integrate the root node freely after the heel is lifted.

8. The state machine control based motion capture optimization method of claim 7, wherein, Calculate the ratio between the change state quantity and the preset reference state quantity as the state ratio; According to the positive correlation of the state ratio, adjust the bias value of the first ratio range, the larger the state ratio, the larger the bias value, the smaller the state ratio, the smaller the bias value; According to the inverse correlation of the state ratio, adjust the reference frame number, the larger the state ratio, the smaller the reference frame number, the smaller the state ratio, the larger the reference frame number.

9. A motion capture optimization system based on state machine control, characterized in that, The processor executes the steps of the action capture optimization method based on the state machine control as claimed in any one of claims 1-8.

10. A storage medium, characterized by The storage medium stores a program, and the program is executed by the processor to realize the steps of the action capture optimization method based on the state machine control as claimed in any one of claims 1-8.

Images