Anmation game character action intelligent capturing method and system
By acquiring signals in real time through motion capture equipment and constructing a motion semantic decoupling model, the semantic and stylistic decoupling of motion capture data is automatically completed, solving the problem of low efficiency in motion capture data processing in existing technologies and realizing efficient clipping repair and stylization transfer.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHAANXI HEZHIMEI ANIMATION IND DEV CO LTD
- Filing Date
- 2026-04-20
- Publication Date
- 2026-07-21
Smart Images
Figure CN122435104A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of motion capture technology, and in particular to a method and system for intelligent motion capture of anime and game characters. Background Technology
[0002] Motion capture technology is a core method for creating animation characters in anime and games. Existing motion capture systems are mainly divided into three categories: optical motion capture uses multiple infrared cameras to capture marker points, generating skeletal joint rotation data through 3D reconstruction and inverse kinematics solving; inertial motion capture uses inertial measurement units to collect acceleration and angular velocity, obtaining joint quaternion sequences through attitude calculation; and monocular video motion capture uses a deep learning attitude estimation network to extract 2D keypoints from an RGB video stream, then uses inverse kinematics to elevate them into 3D skeletal data. The raw motion capture data output by these systems is typically saved in formats such as FBX, BVH, and GLB, containing bone position or rotation information on the timeline. In the actual production process, the raw motion capture data first needs to undergo standardized preprocessing, including skeletal topology alignment to a standard human skeleton, timeline normalization to a fixed frame rate, global coordinate unification to the pelvic local coordinate system, and removal of abrupt frames and interpolation algorithms to fill in missing frames, ultimately obtaining temporal skeletal motion data.
[0003] However, a piece of motion capture data contains both semantic information such as the essential motion patterns of the action, such as the timing of force application and the trajectory of motion, and stylistic details such as the swing amplitude, joint rotation speed, and pause rhythm. Existing methods mix these two types of information in the same set of joint parameters. This means that directly modifying the joint angle during clipping repair will destroy semantic features such as the timing of force application and the trajectory of motion, while adjusting the swing amplitude and rotation speed during stylization transfer will reintroduce clipping or negate the repair effect. Due to the lack of a framework to decouple semantics and style, post-production staff have to iterate repeatedly between clipping repair and style transfer. A complex 10-second action usually requires two to three hours of manual frame-by-frame correction. Therefore, there is an urgent need for an integrated intelligent capture method that can decouple motion capture data into an independent core motion semantic skeleton and replaceable stylistic detail features, and automatically complete clipping repair, timing smoothing, and stylization transfer without destroying semantics. Summary of the Invention
[0004] The purpose of this invention is to provide an intelligent motion capture method and system for anime and game characters, aiming to solve the technical problem in the existing technology of lacking a framework that decouples semantics and style, forcing post-production staff to iterate repeatedly between clipping repair and style transfer, and requiring two to three hours of manual frame-by-frame correction for a complex ten-second action.
[0005] To achieve the above objectives, the present invention employs an intelligent motion capture method for anime and game characters, comprising the following steps: Raw motion signals are collected in real time using motion capture equipment, and then raw realistic motion capture data is obtained through signal processing. The original realistic motion capture data is standardized and preprocessed to obtain temporal skeletal motion data in a unified format; Construct an action semantic decoupling model to decompose the temporal skeletal motion data into a core action semantic skeleton and original style detail features; Based on the core action semantic skeleton, automatic clipping repair and temporal smoothing are performed to obtain a corrected semantic skeleton without clipping or jitter. The modified semantic skeleton is fused with the target anime style detail features to obtain stylized motion capture data and export it in an adapted format.
[0006] In the step of acquiring raw motion signals in real time through motion capture equipment and obtaining raw realistic motion capture data through signal processing: The motion capture device is any one or a combination of an optical motion capture camera, an inertial sensor, or a monocular camera. For optical motion capture, the three-dimensional coordinates of marker points are received from at least 6 infrared cameras. After noise is eliminated by Kalman filtering, the skeletal joint rotation data are generated by inverse kinematics solution. For inertial motion capture, the acceleration and angular velocity data from the inertial measurement unit are received, and joint quaternion data are generated through attitude calculation algorithms. For monocular motion capture, RGB video streams are received, 2D key points are extracted through a deep learning pose estimation network, and then inverse kinematics is used to improve them into 3D skeleton data. The final result is raw realistic motion capture data, which supports temporary export in FBX, BVH, and GLB formats.
[0007] In the step of standardizing and preprocessing the original realistic motion capture data to obtain temporal skeletal motion data in a unified format: The original realistic motion capture data was topologically aligned to a standard 31-joint human skeleton, the time axis was normalized to 30 frames per second, and the global coordinates were unified to a local coordinate system with the pelvis as the origin. Automatically remove abnormal frames with joint angular velocities exceeding 500 degrees per second. For data with no more than 5 consecutive missing frames, use cubic spline interpolation to complete the data, and finally obtain temporal skeletal motion data in a unified format.
[0008] In the step of constructing an action semantic decoupling model, which decomposes the temporal skeletal motion data into a core action semantic skeleton and original style detail features: The action semantic decoupling model adopts an encoder-decoder architecture; The encoder includes a first temporal convolution branch and a second temporal convolution branch. The first branch extracts the low-frequency components of the motion trajectory as the core action semantic skeleton, and the second branch extracts the high-frequency components of the joint angular velocity and swing amplitude as the original style detail features. The encoder also includes a mutual information minimization module, which uses adversarial training to reduce the mutual information between the core action semantic skeleton and the original style detail features to a preset threshold, thereby ensuring the decoupling of semantics and style. The decoder employs a temporal deconvolutional network and a gated fusion mechanism at each layer. The gated fusion mechanism dynamically calculates the fusion weights based on the input core action semantic skeleton and the target anime style detail features to reconstruct the skeleton position sequence.
[0009] In the step of automatically repairing clipping and smoothing the temporal sequence based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter: Import simplified colliders of the target character's body, clothing, and weapons. Each collider is represented as a sphere or capsule, and its parameters are stored in a configuration file. Calculate the position of each bone joint in the core action semantic skeleton and the spatial intersection relationship with the collider frame by frame. When the intersection depth exceeds 3 mm, it is marked as a clipping area, and the clipping joint index and penetration depth are recorded. The joint angles of the bones corresponding to the through-template region are finely adjusted by solving inverse kinematics with motion law constraints. The motion law constraints include: the joint angles after repair should be as close as possible to the joint angles of the original semantic skeleton, and the joint angular velocity after repair should be as small as possible; the single-step correction range of the angle of a single bone should not exceed 15 degrees, and the iterative solution should not exceed 10 times, so as to finally obtain the intermediate skeleton after through-template repair.
[0010] In the step of automatically repairing clipping and smoothing the temporal sequence based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter: An adaptive sliding window weighted average algorithm is used to smooth the joint rotation data of the intermediate skeleton after the through-molding repair. The window size is automatically adjusted according to the joint angular acceleration. The larger the angular acceleration, the larger the window. The minimum window size is 3 frames and the maximum is 7 frames. Pre-detection of action keyframes is performed on the temporal skeletal motion data. Keyframes are defined as the force points and turning points of the action. The keyframe data is not smoothed, but only the non-keyframes are smoothed by applying a Gaussian weighted window, and finally a corrected semantic skeleton without clipping or jitter is obtained.
[0011] In the step of fusing the modified semantic skeleton with the target anime style detail features to obtain stylized motion capture data and exporting it in an adapted format: Load a preset target anime style feature library. The feature library contains four basic target anime style details: anime, cartoon, martial arts, and fantasy. The features of anime style include: joint swing amplitude magnified by 1.3 to 1.5 times, joint rotation speed curve adopting a trapezoidal speed curve, and pause duration extended by 1.2 times. Based on the modified semantic skeleton, the original style detail features in the decoder are replaced with the target anime style detail features. During the fusion process, the motion trajectory, action rhythm and force exertion sequence of the core action semantic skeleton are kept unchanged through a gating fusion mechanism to obtain stylized motion capture data.
[0012] In the process of loading a preset target anime style feature library, which contains detailed features of four basic target anime styles—anime, cartoon, martial arts, and fantasy—the following steps are included: increasing joint swing amplitude by 1.3 to 1.5 times, using a trapezoidal speed curve for joint rotation speed, and extending pause duration by 1.2 times: Receive text descriptions input by the user, which include style adjustment requirements for the range of motion, speed, and pauses; The text description is fed into a pre-trained large language model, which outputs a structured style parameter adjustment vector. The original target anime style detail features are weighted and updated based on the adjustment vector to obtain custom stylized motion capture data that matches the user's description.
[0013] In the step of replacing the original style detail features in the decoder with the target anime style detail features based on the modified semantic skeleton, and maintaining the motion trajectory, action rhythm and force exertion sequence of the core action semantic skeleton unchanged through a gating fusion mechanism during the fusion process to obtain stylized motion capture data: It supports exporting stylized motion capture data in FBX, BVH, GLB, and USD formats, with the export frame rate configurable to 24 frames per second, 30 frames per second, or 60 frames per second. Automatically adapts to the skeletal binding specifications of mainstream game engines such as Unity, Unreal Engine, Blender, and Maya. Adds HumanoidAvatar configuration for Unity and matches APose or TPose redirection for Unreal Engine. At the same time, a metadata file is generated to record the clipping repair record for each frame, the style blending parameters used, and the text or reference image adjustment record, allowing users to make frame-by-frame fine-tuning later.
[0014] This invention also provides an intelligent motion capture system for animation and game characters, including a signal acquisition module, a data preprocessing module, a semantic decoupling module, a repair and smoothing module, and a style fusion and export module; wherein: The signal acquisition module is used to acquire raw motion signals in real time through motion capture equipment, and obtain raw realistic motion capture data through signal processing. The data preprocessing module is used to perform standardized preprocessing on the original realistic motion capture data to obtain temporal skeletal motion data in a unified format. The semantic decoupling module is used to construct an action semantic decoupling model, which decomposes the temporal skeletal motion data into a core action semantic skeleton and original style detail features. The repair and smoothing module is used to perform automatic repair and temporal smoothing processing based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter. The style fusion and export module is used to fuse the modified semantic skeleton with the target animation style details to obtain stylized motion capture data and export it in an adapted format.
[0015] This invention discloses an intelligent motion capture method and system for anime and game characters. The method involves real-time acquisition of raw motion signals using motion capture equipment, followed by signal processing to obtain raw realistic motion capture data. The raw realistic motion capture data is then standardized and preprocessed to obtain temporal skeletal motion data in a unified format. A motion semantic decoupling model is constructed, decomposing the temporal skeletal motion data into a core motion semantic skeleton and original style detail features. Based on the core motion semantic skeleton, automatic clipping repair and temporal smoothing are performed to obtain a corrected semantic skeleton without clipping or jitter. Finally, the corrected semantic skeleton is fused with target anime style detail features to obtain stylized motion capture data, which is then exported in an adapted format. This method decouples motion capture data into a core motion semantic skeleton and style detail features, achieving integrated processing of clipping repair, temporal smoothing, and stylistic transfer. Compared with existing technologies, it offers the following advantages: In the clipping repair stage, inverse kinematic constraints based on the semantic skeleton can automatically eliminate clipping without disrupting the force application timing and motion trajectory, with a single bone correction amplitude not exceeding 15 degrees, avoiding the inefficiency and inconsistency of traditional manual frame-by-frame adjustments; In the temporal smoothing stage, adaptive window smoothing combined with keyframe preservation technology effectively eliminates jitter while maintaining the impact of the action's force application point; In the stylistic transfer stage, a gating fusion mechanism replaces the original style features with target animation style detail features (such as amplitude amplification and speed curve adjustment), fully preserving the motion essence of the core motion semantic skeleton and solving the problem of style transfer disrupting motion logic in existing methods. Overall, this method improves the efficiency of motion capture post-processing by more than 10 times and supports flexible style customization with text descriptions and reference animation guidance. It can directly generate FBX and BVH format data compatible with mainstream engines such as Unity and Unreal, significantly reducing the manual cost and time cycle of animation and game character motion production. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart illustrating the steps of the intelligent motion capture method for anime and game characters according to the present invention.
[0018] Figure 2 This is a flowchart of step S1 of the present invention.
[0019] Figure 3 This is a flowchart of step S2 of the present invention.
[0020] Figure 4 This is a flowchart of step S3 of the present invention.
[0021] Figure 5 This is a flowchart of step S4 of the present invention.
[0022] Figure 6 This is a flowchart of step S5 of the present invention.
[0023] Figure 7 This is a flowchart of steps S501 of the present invention.
[0024] Figure 8This is a flowchart of steps S502 of the present invention.
[0025] Figure 9 This is a structural principle diagram of the intelligent motion capture system for animation and game characters of the present invention.
[0026] Figure 10 This is a schematic diagram of the electronic device of the present invention.
[0027] 601 - Signal Acquisition Module, 602 - Data Preprocessing Module, 603 - Semantic Decoupling Module, 604 - Repair and Smoothing Module, 605 - Style Fusion and Export Module. Detailed Implementation
[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application.
[0029] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.
[0030] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0031] Please see Figures 1-8 This invention provides a method for intelligent motion capture of anime and game characters, comprising the following steps: S1: Raw motion signals are collected in real time through motion capture equipment, and raw realistic motion capture data is obtained through signal processing; In this embodiment, raw motion signals are collected in real time using motion capture equipment, and raw realistic motion capture data is obtained through signal processing.
[0032] S101: The motion capture device is any one or a combination of an optical motion capture camera, an inertial sensor, or a monocular camera; S102: For optical motion capture, the three-dimensional coordinates of marker points collected by at least 6 infrared cameras are received. After noise is eliminated by Kalman filtering, the skeletal joint rotation data are generated by inverse kinematics solution. S103: For inertial motion capture, it receives acceleration and angular velocity data from the inertial measurement unit and generates joint quaternion data through attitude calculation algorithm; S104: For monocular motion capture, RGB video stream is received, 2D key points are extracted through a deep learning pose estimation network, and then inverse kinematics is used to improve it into 3D skeleton data; S105: Finally, the original realistic motion capture data is obtained, and temporary export in FBX, BVH, and GLB formats is supported.
[0033] In this step, motion capture equipment includes optical, inertial, and monocular types, but in actual implementation, any one or more of these can be fused together. Optical motion capture uses at least six infrared cameras to ensure 3D reconstruction accuracy. The state vector of the Kalman filter includes position and velocity, and the observation noise covariance is preset according to the camera resolution. Inertial motion capture uses gradient descent quaternion updates or extended Kalman filters for attitude calculation, and geomagnetic sensors are fused if necessary to suppress drift. The deep learning network for monocular motion capture can be OpenPose or HRNet, and temporal optical flow constraints are used for 3D upscaling. The final exported FBX, BVH, and GLB formats are just examples; other formats that support skeletal animation are also applicable.
[0034] S2: Standardize and preprocess the original realistic motion capture data to obtain temporal skeletal motion data in a unified format; In this embodiment, the original realistic motion capture data is standardized and preprocessed to obtain temporal skeletal motion data in a unified format.
[0035] S201: Perform skeletal topology alignment to a standard 31-joint human skeleton on the original realistic motion capture data, normalize the time axis to 30 frames per second, and unify the global coordinates to a local coordinate system with the pelvis as the origin. S202: Automatically remove abnormal frames with joint angular velocities exceeding 500 degrees per second. For data with no more than 5 consecutive missing frames, use cubic spline interpolation to complete the data, and finally obtain temporal skeletal motion data in a unified format.
[0036] In the standardized preprocessing, skeletal topology alignment uses a mapping table to map different skeletons to a 31-joint standard template, and missing joints are interpolated using the geometric relationships between adjacent joints. The timeline normalization of 30 frames per second is a common game frame rate, which can be adjusted to 24 or 60 frames per second depending on the target platform. Cubic spline interpolation is used to complete data with no more than 5 consecutive missing frames; any frames exceeding 5 are marked as invalid segments and the user is notified. The threshold of 500 degrees per second for removing abruptly changing frames is derived from the physiological limit of human joint angular velocity and can be adjusted appropriately based on the specific character's body shape.
[0037] S3: Construct an action semantic decoupling model to decompose the temporal skeletal motion data into a core action semantic skeleton and original style detail features; In this embodiment, an action semantic decoupling model is constructed to decompose the temporal skeletal motion data into a core action semantic skeleton and original style detail features.
[0038] S301: The action semantic decoupling model adopts an encoder-decoder architecture; S302: The encoder includes a first temporal convolution branch and a second temporal convolution branch. The first branch extracts the low-frequency components of the motion trajectory as the core action semantic skeleton, and the second branch extracts the high-frequency components of the joint angular velocity and swing amplitude as the original style detail features. S303: The encoder also includes a mutual information minimization module, which uses adversarial training to make the mutual information between the core action semantic skeleton and the original style detail features lower than a preset threshold, thereby ensuring the decoupling of semantics and style. S304: The decoder uses a temporal deconvolutional network and a gated fusion mechanism in each layer. The gated fusion mechanism dynamically calculates the fusion weights based on the input core action semantic skeleton and the target animation style detail features to reconstruct the skeleton position sequence.
[0039] The encoder's first branch employs multi-scale temporal convolutional kernels (5, 15, and 31 frames) to capture low-frequency components of motion trajectories across different time spans. The kernel size is related to the motion cycle, with longer kernels suitable for slow motion. The second branch uses small convolutional kernels (3 frames) to extract high-frequency oscillations. The mutual information minimization module is implemented through adversarial training, with a preset threshold ε=0.05nats based on experimental experience, allowing for fluctuations of ±0.01. In the decoder's gating fusion mechanism, the gating units use Sigmoid activation, with fusion weights between 0 and 1. During training, a weighted sum of reconstruction loss and semantic preservation loss is used.
[0040] S4: Based on the core action semantic skeleton, perform automatic clipping repair and temporal smoothing to obtain a corrected semantic skeleton without clipping or jitter. In this embodiment, based on the core action semantic skeleton, automatic clipping repair and timing smoothing are performed to obtain a corrected semantic skeleton without clipping or jitter.
[0041] S401: Import simplified colliders of the target character's body, clothing, and weapons. Each collider is represented as a sphere or capsule, and its parameters are stored in a configuration file. S402: Calculate the position of each bone joint in the core action semantic skeleton and the spatial intersection relationship with the collider frame by frame. When the intersection depth exceeds 3 mm, it is marked as a clipping area, and the clipping joint index and penetration depth are recorded. S403: Fine-tune the joint angles of the bones corresponding to the through-template region by solving inverse kinematics with motion law constraints. The motion law constraints include: the repaired joint angles should be as close as possible to the joint angles of the original semantic skeleton, and the repaired joint angular velocity should be as small as possible; the single-step correction range of the angle of a single bone should not exceed 15 degrees, and the iterative solution should not exceed 10 times, finally obtaining the intermediate skeleton after through-template repair.
[0042] S404: An adaptive sliding window weighted average algorithm is used to smooth the joint rotation data of the intermediate skeleton after the through-molding repair. The window size is automatically adjusted according to the joint angular acceleration. The larger the angular acceleration, the larger the window. The minimum window size is 3 frames and the maximum is 7 frames. S405: Pre-detect keyframes of the temporal skeleton motion data. Keyframes are defined as the force points and turning points of the action. The keyframe data is not smoothed. Only the non-keyframes are smoothed by applying a Gaussian weighted window, and finally a corrected semantic skeleton without clipping or jitter is obtained.
[0043] In this process, simplified colliders are represented by spheres or capsules to reduce computational load, with their parameters (radius, half-height) automatically estimated from the target character model or manually configured. The depth threshold for clipping detection, 3 mm, is the minimum perceptible penetration by the human eye and can be adjusted to 5 mm depending on model accuracy. In the inverse kinematics solution's motion constraints, weight coefficients λ1=0.8 and λ2=0.2 ensure semantic fidelity priority, generally converging within 10 iterations. The adaptive sliding window's angular acceleration mapping formula is: window size = min(7, max(3, floor(α_max / 100) + 3)), where α_max is in ° / s². Keyframe detection employs dual criteria: angular acceleration crossing zero points and velocity direction change points.
[0044] S5: The modified semantic skeleton is fused with the target anime style detail features to obtain stylized motion capture data and the adapted format is exported.
[0045] In this embodiment, the modified semantic skeleton is fused with the target anime style detail features to obtain stylized motion capture data and export it in an adapted format.
[0046] S501: Load the preset target anime style feature library. The feature library contains four basic target anime style details: anime, cartoon, martial arts, and fantasy. The features of the anime style include: joint swing amplitude magnified by 1.3 to 1.5 times, joint rotation speed curve adopts a trapezoidal speed curve, and pause duration extended by 1.2 times. S5011: Receive a text description input by the user, the text description including style adjustment requirements for the amplitude, speed, and pauses of the movements; S5012: Feed the text description into a pre-trained large language model, which outputs a structured style parameter adjustment vector; S5013: The original target animation style detail features are weighted and updated according to the adjustment vector to obtain custom stylized motion capture data that conforms to the user description.
[0047] S502: Based on the modified semantic skeleton, the original style detail features in the decoder are replaced with the target animation style detail features. During the fusion process, the motion trajectory, action rhythm and force exertion sequence of the core action semantic skeleton are kept unchanged through the gating fusion mechanism to obtain stylized motion capture data.
[0048] S5021: Supports exporting stylized motion capture data in FBX, BVH, GLB, and USD formats. The export frame rate can be configured to 24 frames per second, 30 frames per second, or 60 frames per second. S5022: Automatically adapts to the skeletal binding specifications of mainstream game engines such as Unity, Unreal Engine, Blender, and Maya. For Unity, it adds HumanoidAvatar configuration, and for Unreal Engine, it matches APose or TPose redirection. S5023: Simultaneously generates a metadata file that records the clipping repair record for each frame, the style blending parameters used, and the text or reference image adjustment record, supporting users to perform frame-by-frame fine-tuning later.
[0049] The preset style library's anime style parameters (amplitude 1.3~1.5 times, trapezoidal velocity curve, pause duration 1.2 times) are derived from statistical analysis of typical anime movements; other style parameters can be similarly preset. When adjusting text descriptions, the large language model outputs an adjustment vector with the same dimension as the style features, with the weighting coefficient α ranging from 0 to 1, defaulting to 0.5. When referencing animation frames for guidance, the style feature mapping network is a 3-layer fully connected network with 256 hidden layer dimensions. The export format supports USD to adapt to film and television workflows, and the metadata file is in JSON format for easy parsing by later tools.
[0050] Taking an actor's "rapid right-hand punch" action as an example, 12 infrared optical motion capture cameras collected 120 frames of raw signal at 60 frames per second for 2 seconds. After noise removal by Kalman filtering, inverse kinematics was used to generate raw realistic motion capture data of a 31-joint human skeleton, which was temporarily exported as an FBX file. Then, standardization preprocessing was performed: the raw skeleton was aligned to a standard 31-joint template using a mapping table, the coordinates were unified to the pelvic local coordinate system, the time axis was normalized to 30 frames per second, and jump frames with joint angular velocities exceeding 500 degrees per second were removed. Data with no more than 5 consecutive missing frames was completed using cubic spline interpolation, resulting in uniform temporal skeletal motion data. This data was input into the encoder of the motion semantic decoupling model. The first branch of the encoder used multi-scale temporal convolution with kernel sizes of 5, 15, and 31 frames to extract the low-frequency components of the motion trajectory as the core motion semantic skeleton (preserving the punch's force timing and motion trajectory). The second branch used convolution with a kernel size of 3 frames to extract the high-frequency components of the joint angular velocity and swing amplitude as the core motion semantic skeleton. The original style details (such as the subtle tremors of arm swings) are preserved, and the mutual information minimization module ensures decoupling by reducing the mutual information between the two to below 0.05 NATS through adversarial training. Automatic clipping repair is performed based on the core motion semantic skeleton. The capsule-shaped colliders of the target anime character's hand and body are imported, and the intersection depth of the fist and chest colliders is calculated frame by frame. When the intersection depth exceeds 3 mm, the shoulder, elbow, and wrist joint angles are finely adjusted using inverse kinematics with motion constraints. The constraints are that the corrected angles are close to the original semantic skeleton and the angular velocity is as small as possible, with a single-step correction not exceeding 15 degrees. After 8 iterations, the intermediate skeleton after clipping repair is obtained. Then, an adaptive sliding window is used for smoothing. The window size is automatically adjusted to 5 frames based on angular acceleration. A Gaussian weighted window is applied to non-keyframes, and the punching point is retained as a keyframe without smoothing. Finally, a corrected semantic skeleton without clipping or tremors is obtained. This corrected semantic skeleton is then compared with preset anime style details (joint swing amplitude increased by 1.4 times, joint rotation speed curve changed to a trapezoidal speed curve, and pause duration extended by 1).The input decoder (2x) uses a gated fusion mechanism to replace the original style features with the target style features, while maintaining the punch trajectory, rhythm, and timing of force application, thus reconstructing stylized motion capture data. Furthermore, this method allows users to adjust the style via text descriptions (e.g., "Make the movement more exaggerated, increase arm swing amplitude by 20%)." After parsing by a large language model, adjustment vectors are generated to weighted update the style features, resulting in a custom style. Finally, the stylized motion capture data is exported, supporting multiple formats such as FBX, BVH, GLB, and USD. This embodiment exports in FBX format, with a frame rate configured at 30 frames per second, automatically adapting to the Humanoid Avatar skeletal binding specification of the Unity engine (or Unreal Engine's A Pose or T Pose redirection, or Blender / Maya's skeletal constraints, depending on the target engine). A JSON metadata file is also generated to record the clipping repair angle and style fusion parameters for each frame, completing the entire intelligent processing from the original motion capture signal to a 2D-style boxing animation.
[0051] The mutual information minimization module is implemented through adversarial training, specifically as follows: The core action semantic skeleton output by the first branch of the encoder records the low-frequency motion trajectory of the action, such as the overall movement path of the arm from charging to striking when punching; the original style detail features output by the second branch record the high-frequency swing details, such as slight tremors or speed fluctuations in arm movement. To force the two to be independent, a discriminator network is introduced. This discriminator receives semantic and style features as input at the same time point each time and determines whether they come from the same natural pairing of actions. During training, two types of samples are constructed: the first type is the semantic and style pairing actually output by the encoder (called the true pairing), and the second type is the style features randomly shuffled on the time axis and re-paired with the semantics (called the false pairing). The discriminator's task is to distinguish between true and false pairings, while the encoder's task is to generate features that are difficult for the discriminator to distinguish. Through this adversarial game, the encoder is forced to learn to remove style-related information from semantic features and simultaneously remove semantic-related information from style features. Every 500 action segments trained, the residual correlation between the current semantics and style is automatically detected. If the correlation is still significant, the penalty for adversarial training is increased; if they are largely independent, the current parameters are maintained and fine-tuned. The final encoder can output mutually independent semantic skeletons and style features, ensuring that subsequent clipping repair does not change the action semantics and style transfer does not destroy the repair effect.
[0052] Corresponding to the aforementioned embodiments of the intelligent motion capture method for anime and game characters, this application also provides embodiments of an intelligent motion capture system for anime and game characters.
[0053] Figure 9 This is a block diagram illustrating an intelligent motion capture system for anime and game characters according to an exemplary embodiment. (Refer to...) Figure 9 The system may include a signal acquisition module, a data preprocessing module, a semantic decoupling module, a repair and smoothing module, and a style fusion and export module, wherein: The signal acquisition module is used to acquire raw motion signals in real time through motion capture equipment, and obtain raw realistic motion capture data through signal processing. The data preprocessing module is used to perform standardized preprocessing on the original realistic motion capture data to obtain temporal skeletal motion data in a unified format. The semantic decoupling module is used to construct an action semantic decoupling model, which decomposes the temporal skeletal motion data into a core action semantic skeleton and original style detail features. The repair and smoothing module is used to perform automatic repair and temporal smoothing processing based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter. The style fusion and export module is used to fuse the modified semantic skeleton with the target animation style details to obtain stylized motion capture data and export it in an adapted format.
[0054] In this embodiment, the signal acquisition module is used to acquire raw motion signals in real time through motion capture equipment and obtain raw realistic motion capture data through signal processing; the data preprocessing module is used to perform standardized preprocessing on the raw realistic motion capture data to obtain temporal skeletal motion data in a unified format; the semantic decoupling module is used to construct a motion semantic decoupling model to decompose the temporal skeletal motion data into a core motion semantic skeleton and original style detail features; the repair and smoothing module is used to perform automatic clipping repair and temporal smoothing processing based on the core motion semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter; the style fusion and export module is used to fuse the corrected semantic skeleton with the target animation style detail features to obtain stylized motion capture data and export it in an adapted format.
[0055] Regarding the system in the above embodiments, the specific manner in which each module performs its operations has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0056] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0057] Accordingly, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the above-described intelligent motion capture method for anime and game characters. Figure 10 The diagram shown is a hardware structure diagram of any device with data processing capabilities, used in an intelligent motion capture system for anime and game characters according to an embodiment of the present invention. (Except for...) Figure 10 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.
[0058] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the above-described intelligent motion capture method for anime and game characters. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.
[0059] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.
[0060] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.
Claims
1. A method for intelligent motion capture of anime and game characters, characterized in that, Includes the following steps: Raw motion signals are collected in real time using motion capture equipment, and then raw realistic motion capture data is obtained through signal processing. The original realistic motion capture data is standardized and preprocessed to obtain temporal skeletal motion data in a unified format; Construct an action semantic decoupling model to decompose the temporal skeletal motion data into a core action semantic skeleton and original style detail features; Based on the core action semantic skeleton, automatic clipping repair and temporal smoothing are performed to obtain a corrected semantic skeleton without clipping or jitter. The modified semantic skeleton is fused with the target anime style detail features to obtain stylized motion capture data and export it in an adapted format.
2. The intelligent motion capture method for anime and game characters as described in claim 1, characterized in that, In the step of acquiring raw motion signals in real time using motion capture equipment and obtaining raw realistic motion capture data through signal processing: The motion capture device is any one or a combination of an optical motion capture camera, an inertial sensor, or a monocular camera. For optical motion capture, the three-dimensional coordinates of marker points are received from at least 6 infrared cameras. After noise is eliminated by Kalman filtering, the skeletal joint rotation data are generated by inverse kinematics solution. For inertial motion capture, the acceleration and angular velocity data from the inertial measurement unit are received, and joint quaternion data are generated through attitude calculation algorithms. For monocular motion capture, RGB video streams are received, 2D key points are extracted through a deep learning pose estimation network, and then inverse kinematics is used to improve them into 3D skeleton data. The final result is raw realistic motion capture data, which supports temporary export in FBX, BVH, and GLB formats.
3. The intelligent motion capture method for anime and game characters as described in claim 1, characterized in that, In the step of standardizing and preprocessing the original realistic motion capture data to obtain temporal skeletal motion data in a unified format: The original realistic motion capture data was topologically aligned to a standard 31-joint human skeleton, the time axis was normalized to 30 frames per second, and the global coordinates were unified to a local coordinate system with the pelvis as the origin. Automatically remove abnormal frames with joint angular velocities exceeding 500 degrees per second. For data with no more than 5 consecutive missing frames, use cubic spline interpolation to complete the data, and finally obtain temporal skeletal motion data in a unified format.
4. The intelligent motion capture method for anime and game characters as described in claim 1, characterized in that, In the step of constructing an action semantic decoupling model, decomposing the temporal skeletal motion data into a core action semantic skeleton and original style detail features: The action semantic decoupling model adopts an encoder-decoder architecture; The encoder includes a first temporal convolution branch and a second temporal convolution branch. The first branch extracts the low-frequency components of the motion trajectory as the core action semantic skeleton, and the second branch extracts the high-frequency components of the joint angular velocity and swing amplitude as the original style detail features. The encoder also includes a mutual information minimization module, which uses adversarial training to reduce the mutual information between the core action semantic skeleton and the original style detail features to a preset threshold, thereby ensuring the decoupling of semantics and style. The decoder employs a temporal deconvolutional network and a gated fusion mechanism at each layer. The gated fusion mechanism dynamically calculates the fusion weights based on the input core action semantic skeleton and the target anime style detail features to reconstruct the skeleton position sequence.
5. The intelligent motion capture method for anime and game characters as described in claim 1, characterized in that, In the step of automatically repairing clipping and smoothing the temporal sequence based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter: Import simplified colliders of the target character's body, clothing, and weapons. Each collider is represented as a sphere or capsule, and its parameters are stored in a configuration file. Calculate the position of each bone joint in the core action semantic skeleton and the spatial intersection relationship with the collider frame by frame. When the intersection depth exceeds 3 mm, it is marked as a clipping area, and the clipping joint index and penetration depth are recorded. The joint angles of the bones corresponding to the through-template region are finely adjusted by solving inverse kinematics with motion law constraints. The motion law constraints include: the joint angles after repair should be as close as possible to the joint angles of the original semantic skeleton, and the joint angular velocity after repair should be as small as possible; the single-step correction range of the angle of a single bone should not exceed 15 degrees, and the iterative solution should not exceed 10 times, so as to finally obtain the intermediate skeleton after through-template repair.
6. The intelligent motion capture method for anime and game characters as described in claim 5, characterized in that, In the step of automatically repairing clipping and smoothing the temporal sequence based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter: An adaptive sliding window weighted average algorithm is used to smooth the joint rotation data of the intermediate skeleton after the through-molding repair. The window size is automatically adjusted according to the joint angular acceleration. The larger the angular acceleration, the larger the window. The minimum window size is 3 frames and the maximum is 7 frames. Pre-detection of action keyframes is performed on the temporal skeletal motion data. Keyframes are defined as the force points and turning points of the action. The keyframe data is not smoothed, but only the non-keyframes are smoothed by applying a Gaussian weighted window, and finally a corrected semantic skeleton without clipping or jitter is obtained.
7. The intelligent motion capture method for animation and game characters as described in claim 1, characterized in that, In the step of fusing the modified semantic skeleton with the target animation style details to obtain stylized motion capture data and exporting it in an adapted format: Load a preset target anime style feature library. The feature library contains four basic target anime style details: anime, cartoon, martial arts, and fantasy. The features of anime style include: joint swing amplitude magnified by 1.3 to 1.5 times, joint rotation speed curve adopting a trapezoidal speed curve, and pause duration extended by 1.2 times. Based on the modified semantic skeleton, the original style detail features in the decoder are replaced with the target anime style detail features. During the fusion process, the motion trajectory, action rhythm and force exertion sequence of the core action semantic skeleton are kept unchanged through a gating fusion mechanism to obtain stylized motion capture data.
8. The intelligent motion capture method for anime and game characters as described in claim 7, characterized in that, In the process of loading a preset target anime style feature library, which contains detailed features of four basic target anime styles—anime, cartoon, martial arts, and fantasy—the following steps are taken: increasing joint swing amplitude by 1.3 to 1.5 times, using a trapezoidal speed curve for joint rotation speed, and extending pause duration by 1.2 times: Receive text descriptions input by the user, which include style adjustment requirements for the range of motion, speed, and pauses; The text description is fed into a pre-trained large language model, which outputs a structured style parameter adjustment vector. The original target anime style detail features are weighted and updated based on the adjustment vector to obtain custom stylized motion capture data that matches the user's description.
9. The intelligent motion capture method for animation and game characters as described in claim 7, characterized in that, In the step of replacing the original style detail features in the decoder with the target anime style detail features based on the modified semantic skeleton, and maintaining the motion trajectory, action rhythm and force exertion sequence of the core action semantic skeleton unchanged through a gating fusion mechanism during the fusion process to obtain stylized motion capture data: It supports exporting stylized motion capture data in FBX, BVH, GLB, and USD formats, with the export frame rate configurable to 24 frames per second, 30 frames per second, or 60 frames per second. Automatically adapts to the skeletal binding specifications of mainstream game engines such as Unity, Unreal Engine, Blender, and Maya. Adds HumanoidAvatar configuration for Unity and matches APose or TPose redirection for Unreal Engine. At the same time, a metadata file is generated to record the clipping repair record for each frame, the style blending parameters used, and the text or reference image adjustment record, allowing users to make frame-by-frame fine-tuning later.
10. A smart motion capture system for anime and game characters, employing the smart motion capture method for anime and game characters as described in claim 1, characterized in that, It includes a signal acquisition module, a data preprocessing module, a semantic decoupling module, a repair and smoothing module, and a style fusion and export module; among which: The signal acquisition module is used to acquire raw motion signals in real time through motion capture equipment, and obtain raw realistic motion capture data through signal processing. The data preprocessing module is used to perform standardized preprocessing on the original realistic motion capture data to obtain temporal skeletal motion data in a unified format. The semantic decoupling module is used to construct an action semantic decoupling model, which decomposes the temporal skeletal motion data into a core action semantic skeleton and original style detail features. The repair and smoothing module is used to perform automatic repair and temporal smoothing processing based on the core action semantic skeleton to obtain a corrected semantic skeleton without clipping or jitter. The style fusion and export module is used to fuse the modified semantic skeleton with the target animation style details to obtain stylized motion capture data and export it in an adapted format.