Intelligent Animation Modeling Method and System Based on 3D Stereo Technology

Abstract

The present invention discloses an intelligent animation modeling method and system based on three-dimensional technology, specifically relating to the field of animation modeling technology; by acquiring the action data of performers in real time, preprocessing the captured data, and combining with bone modeling technology to complete the mapping and optimization of action data and three-dimensional characters; for multi-actor interaction scenarios, analyzing the abnormal changes in the occlusion rate and mis-matching rate of marker points to evaluate the accuracy of the system for multi-character action tracking; after the capture system reaches the expected accuracy, simulating and testing the generated three-dimensional animation, and optimizing the animation data in combination with a physics engine to make it conform to physical laws, exporting the optimized animation in a compatible format, and completing the integration of the character and the scene through real-time rendering technology to generate high-quality three-dimensional animation effects, significantly improving the action capture accuracy and animation generation quality in multi-character complex interaction scenarios, and realizing smooth, accurate and realistic three-dimensional character animation modeling.

Description

Technical Field

[0001] The present invention relates to the technical field of animation modeling, and particularly to an intelligent animation modeling method and system based on three-dimensional technology. Background Art

[0002] Intelligent animation modeling based on three-dimensional technology is an intelligent modeling method based on three-dimensional (3D) technology. It greatly simplifies the complex animation production process through algorithms and automated tools, improving efficiency and accuracy. This technology combines the principles of computer graphics, artificial intelligence (AI), and animation production, making animation modeling no longer completely dependent on manual operations, but more relying on data-driven generation methods, such as learning to generate animation models from existing data through machine learning.

[0003] For example, in traditional animation modeling, to create a character, each frame of the character's actions needs to be hand-drawn, while intelligent animation modeling based on three-dimensional technology can obtain the real action data of actors through motion capture technology and automatically optimize and bind the bones in combination with AI, thereby generating smooth and realistic three-dimensional character animations. This method has been widely used in film and game production. For example, motion capture technology was used in "Avatar" to generate character actions, and intelligent algorithms were combined to improve the facial expressions and body dynamic models of the characters, thereby achieving realistic three-dimensional animation effects.

[0004] The prior art has the following deficiencies:

[0005] During the motion capture process, occlusion, overlapping actions, or marker point confusion among multiple actors can cause the device to have difficulty accurately tracking the independent actions of each character. For example, in simulated hugging, wrestling, or intensive interaction scenarios, the capture system may not be able to distinguish the movement trajectories of different actors, resulting in the loss, misbinding, or merging errors of action data, and thus the actions appear chaotic or unnatural when generating animations. In addition, due to marker point confusion, the action trajectories of different actors may be wrongly merged, resulting in the character behaviors generated by intelligent animation modeling being seriously inconsistent with expectations. This thus destroys the interaction logic between characters, especially in scenarios that require highly precise interaction (such as movie special effects or training simulations), directly rendering the output animations unusable. Summary of the Invention

[0006] The purpose of the present invention is to provide an intelligent animation modeling method and system based on three-dimensional technology to solve the deficiencies in the background art.

[0007] To achieve the above purpose, the present invention provides the following technical solution: An intelligent animation modeling method based on three-dimensional technology, comprising the following steps:

[0008] S1: Use a multi-view motion capture device to obtain the performer's motion data in real time, including setting marker points on the key parts of the performer and recording their positions, rotations, and motion trajectories;

[0009] S2: After preprocessing the captured motion data, create a skeletal model of the target 3D character, including joint positions and hierarchical relationships, and map the preprocessed motion data onto the skeletal model through scale adjustment and weight optimization;

[0010] S3: Use artificial intelligence to optimize the skeletal animation, including motion smoothing and keyframe extraction, and at the same time generate character facial animations to achieve natural matching of expressions and actions. Complete the un-captured part of the motion through deep learning and generate smooth transition animations;

[0011] S4: For multi-actor interaction scenarios, analyze the abnormal changes in the marker occlusion rate and marker mis-matching rate during the marker tracking process of the capture system, and evaluate the motion tracking ability of the capture system for multiple characters;

[0012] S5: When the capture system can accurately distinguish and record the independent motion trajectories of each actor, simulate and test the generated 3D animation, discover and correct potential problems, including penetration and joint over-limit, and optimize the animation data in combination with the physics engine to make it conform to physical laws;

[0013] S6: Export the optimized animation in a compatible format and integrate it into the target scene. Complete the integration of the character and the scene through real-time rendering technology, add lights, materials, and special effects to generate the final high-quality animation effect.

[0014] Preferably, in S2, according to the body type of the 3D character, adjust the bone length to match the target model. Specifically: Use the automatic Retargeting algorithm to scale the motion ratio in the captured data to fit the bones of the character, bind the cleaned motion data to the skeletal model to generate a preliminary animation, use the Retargeting technology to map the preprocessed motion data to the joints of the character's bones, automatically adjust the bone ratio differences of different characters, and set the weight distribution for the bone binding of the character model. The weights at the joint parts need to be evenly distributed.

[0015] Preferably, in S4, after analyzing the abnormal changes in the marker occlusion rate during the marker tracking process of the capture system, generate a marker occlusion rate fluctuation index. The method for obtaining the marker occlusion rate fluctuation index is:

[0016] Construct a time series of the marker occlusion rate , where represents the marker occlusion rate of the Nth frame, and the value range is 0 ≤ ≤10, where N is the total number of frames in the time series. Set the length of the sliding window to W, that is, the number of frames included in the window. For the t-th frame, the sliding average is calculated as follows: ; when t < W, the number of frames included in the window is t, is the occlusion rate of the i-th frame; for the t-th frame, the occlusion rate and the sliding average The formula for the absolute deviation is: ; is the deviation value of the t-th frame, indicating the difference between the current occlusion rate and the sliding average trend. Calculate the marker point occlusion rate fluctuation index, and the expression is: ; where CHK is the marker point occlusion rate fluctuation index, and N is the total number of frames, representing the length of the time series of occlusion rate data.

[0017] Preferably, in S4, after analyzing the abnormal change of the marker point mismatching rate during the marker point tracking process of the capture system, a marker point mismatching rate abnormal index is generated. The method for obtaining the marker point mismatching rate abnormal index is:

[0018] Construct a time series of the marker point mismatching rate , where represents the mismatching rate of the G-th frame, calculate the mean value of the sequence and subtract the mean value from each point to obtain a zero-mean sequence ; s is the number of frames. Perform FFT transformation on the mismatching rate sequence to obtain the frequency domain representation ; ; where is the complex representation of the frequency component k, including amplitude and phase information, is the kernel function of the Fourier transform; for each frequency component , calculate the amplitude: ; where is the amplitude of the frequency component k, and are The real and imaginary parts of; determine the high-frequency range , extract the amplitude set of high-frequency components ; According to the amplitude of the high-frequency components, calculate the marker point mismatching rate abnormal index, and the expression is: ; where FGH is the marker point mismatching rate abnormal index.

[0019] Preferably, the marker point occlusion rate fluctuation index and the marker point mis-matching rate anomaly index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of a machine learning model. The machine learning model takes the prediction of the action tracking accuracy value label of the multi-role by the capture system for each group of comprehensive feature vectors as the prediction target, and takes minimizing the sum of the prediction errors of the action tracking accuracy value labels of all capture systems for the multi-role as the training target to train the machine learning model until the sum of the prediction errors reaches convergence and then stops the model training. The action tracking accuracy value of the capture system for the multi-role is determined according to the model output result, where the machine learning model is a polynomial regression model.

[0020] Preferably, the obtained action tracking accuracy value of the capture system for the multi-role is compared with the action tracking accuracy reference threshold preset according to historical data. If the action tracking accuracy value of the capture system for the multi-role is greater than or equal to the preset action tracking accuracy reference threshold, it indicates that the action tracking accuracy of the capture system for the multi-role is high, and at this time, an action tracking accurate signal is generated, that is, the action tracking ability of the capture system for the multi-role is strong; if the action tracking accuracy value of the capture system for the multi-role is less than the preset action tracking accuracy reference threshold, it indicates that the action tracking accuracy of the capture system for the multi-role is low, and at this time, an action tracking inaccurate signal is generated, that is, the action tracking ability of the capture system for the multi-role is weak.

[0021] Preferably, in S5, when the capture system can accurately distinguish and record the independent action trajectories of each actor, that is, the action tracking accuracy value generated by the capture system for the multi-role within a fixed time period is greater than or equal to the preset action tracking accuracy reference threshold, the generated three-dimensional animation is simulated and tested, and the action tracking accuracy values greater than or equal to the preset action tracking accuracy reference threshold generated in subsequent fixed time periods are collected, and a data set is established. The mean and standard deviation of the data set are calculated, and after analysis, potential problems are predicted in advance according to the analysis results.

[0022] Preferably, if the mean value of the action tracking accuracy values in the data set is greater than or equal to the reference threshold of the mean value of the action tracking accuracy values, and the standard deviation of the action tracking accuracy values is less than the reference threshold of the standard deviation of the action tracking accuracy values, it indicates that the capture system can accurately distinguish and record the independent action trajectories of each role, the generated three-dimensional animation has a small error and conforms to the physical laws, and the capture device and algorithm are continued to be maintained under the current configuration;

[0023] If the mean value of the action tracking accuracy is greater than or equal to the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the action tracking accuracy, it indicates that there are still fluctuations in the capture system during some periods, the action tracking ability is unstable, affecting the local quality of the 3D animation, and it is necessary to optimize the robustness and anti-occlusion ability of the capture algorithm and enhance the adaptability to dynamic changing scenes;

[0024] If the mean value of the action tracking accuracy is less than the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the action tracking accuracy, it indicates that the overall performance of the capture system is insufficient, and at the same time shows instability. There are serious defects in the action tracking ability of the capture system for multiple characters, and comprehensive optimization is carried out from the aspects of device configuration and algorithm;

[0025] If the mean value of the action tracking accuracy is less than the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is less than the reference threshold of the standard deviation of the action tracking accuracy, it indicates that the performance of the capture system is poor but stable, but it cannot meet the high-precision requirements, and it is necessary to optimize the device and algorithm to improve the accuracy of action tracking.

[0026] The present invention also provides an intelligent animation modeling system based on 3D stereoscopic technology, including an action capture module, a data preprocessing and skeleton modeling module, an AI animation optimization module, a multi-character tracking and evaluation module, an animation simulation and physical optimization module, and an animation rendering and integration module:

[0027] Action capture module: Utilize multi-view motion capture devices to obtain the action data of the performer in real time, including setting marker points at the key parts of the performer and recording their positions, rotations, and motion trajectories;

[0028] Data preprocessing and skeleton modeling module: After preprocessing the captured action data, create a skeleton model of the target 3D character, including joint positions and hierarchical relationships, and map the preprocessed action data to the skeleton model through scale adjustment and weight optimization;

[0029] AI animation optimization module: Use artificial intelligence to optimize the skeleton animation, including action smoothing and key frame extraction, and at the same time generate character facial animations to achieve natural matching of expressions and actions, and complete the un-captured part of the action through deep learning to generate smooth transition animations;

[0030] Multi-character tracking and evaluation module: For multi-actor interaction scenarios, analyze the abnormal changes in the marker point occlusion rate and marker point mis-matching rate during the marker point tracking process of the capture system, and evaluate the action tracking ability of the capture system for multiple characters;

[0031] Animation Simulation and Physical Optimization Module: When the capture system can accurately distinguish and record the independent motion trajectories of each actor, simulate and test the generated 3D animations, discover and correct potential problems, including penetration and joint overrun, and optimize the animation data in combination with the physics engine to make it conform to physical laws;

[0032] Animation Rendering and Integration Module: Export the optimized animations in a compatible format and integrate them into the target scene. Complete the integration of the character and the scene through real-time rendering technology, add lights, materials, and special effects to generate the final high-quality animation effect.

[0033] In the above technical solutions, the technical effects and advantages provided by the present invention are as follows:

[0034] 1. Through innovative multi-view capture technology, data preprocessing, artificial intelligence optimization, anomaly analysis, and physics engine integration, the present invention significantly improves the motion capture and 3D animation modeling capabilities in complex multi-character interaction scenarios. The capture performance is quantified using the marker occlusion rate fluctuation index and the mis-matching rate anomaly index, and the motion tracking accuracy of the capture system is dynamically evaluated and optimized by driving the machine learning model with the comprehensive feature vector. This method can quickly discover and correct potential problems, including marker confusion, motion errors, and data loss, making the capture system more stable and efficient in complex multi-character scenarios and laying a solid foundation for generating high-precision 3D animations.

[0035] 2. Combining deep learning and physics engine technology, the present invention achieves smooth motion, key frame optimization, and natural transitions in the animation generation process, supporting the high-precision presentation of complex expressions and details. Through real-time analysis and optimization decisions on the captured data, it ensures that the animation output conforms to physical laws and has the minimum error, and finally completes the high-quality integration of the character and the scene through real-time rendering technology. The overall technical system improves the accuracy and robustness of motion capture, effectively solves the problems of chaotic and unnatural motions in multi-character scenarios in the existing technology, and provides strong technical support for applications such as film and television special effects, virtual reality, and game animations. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0037] Figure 1 It is the method flow chart of the present invention.

[0038] Figure 2 It is the system module diagram of the present invention. Detailed Embodiments

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0040] Example 1, please refer to Figure 1 As shown, the intelligent animation modeling method based on three-dimensional technology in this embodiment includes the following steps:

[0041] S1: Use a multi-view motion capture device to obtain the action data of the performer in real time, including setting marker points on the key parts of the performer and recording their positions, rotations, and movement trajectories;

[0042] S2: After preprocessing the captured action data, create a skeletal model of the target three-dimensional character, including joint positions and hierarchical relationships, and map the preprocessed action data onto the skeletal model through scale adjustment and weight optimization;

[0043] S3: Use artificial intelligence to optimize the skeletal animation, including action smoothing and key frame extraction, and at the same time generate character facial animations to achieve natural matching of expressions and actions, and complete the un-captured part of the action through deep learning to generate smooth transition animations;

[0044] S4: For multi-actor interaction scenarios, analyze the abnormal changes in the marker occlusion rate and marker mis-matching rate during the marker tracking process of the capture system, and evaluate the action tracking ability of the capture system for multiple characters;

[0045] S5: When the capture system can accurately distinguish and record the independent action trajectories of each actor, simulate and test the generated three-dimensional animation, discover and correct potential problems, including penetration and joint overrun, and optimize the animation data in combination with the physics engine to make it conform to physical laws;

[0046] S6: Export the optimized animation in a compatible format and integrate it into the target scene, complete the integration of the character and the scene through real-time rendering technology, add lights, materials, and special effects to generate the final high-quality animation effect.

[0047] Real-time acquisition of the performer's motion data using a multi-view motion capture device. The specific implementation steps include: Device configuration and layout: Select a suitable multi-view motion capture device (such as an optical capture system, an inertial capture system, or a hybrid capture system) to ensure coverage of all angles of the capture scene. Arrange multiple high-resolution cameras in the capture venue, with a certain overlapping area between the cameras to ensure that the marker points are not blocked by a single view. Calibrate the device to unify the coordinate systems of the multiple cameras and ensure the consistency and accuracy of the captured data. Selection and setting of marker points: Selection of key parts: According to the anatomical structure of the performer, set reflective or active marker points at key parts of the body, such as joints (shoulder, elbow, knee), limbs, torso, head, and fingers. Quantity and distribution of marker points: According to the complexity of the action and the requirements of the 3D model, distribute a sufficient number of marker points to ensure that the motion trajectories of each joint and part can be accurately captured.

[0048] The performer performs the action, and the motion capture device captures the 3D positions and rotation angles of each marker point in real time at a high frame rate (usually 60 fps or higher). The capture system calculates the spatial coordinates of the marker points through multi-view data fusion and generates a complete motion trajectory. Synchronously record the time stamps and trajectory data to provide a time series basis for subsequent processing. During the multi-view fusion process, the algorithm automatically detects the occluded marker points and uses the data from other views to supplement the occlusion information. Adjust the camera layout or introduce dynamic occlusion compensation technology to reduce data loss caused by the overlapping or superposition of the performer's actions.

[0049] S2: After preprocessing the captured motion data, create a skeletal model of the target 3D character, including joint positions and hierarchical relationships, and map the preprocessed motion data onto the skeletal model through scale adjustment and weight optimization.

[0050] Preprocess the captured motion data, use filtering algorithms (such as Kalman filtering or low-pass filtering) to eliminate jitter and noise in the captured data. For short-term abnormal data (such as marker point offsets), use interpolation methods to repair. For occluded or missing marker point parts, restore the missing data through trajectory prediction algorithms (such as deep learning-based temporal prediction models). Ensure the time continuity and spatial integrity of the motion trajectory. Convert the captured motion data into a standardized format (such as FBX or BVH) to facilitate compatibility with different animation software and tools. Align the data on the time axis to ensure time synchronization between different segments.

[0051] Define the joint positions (such as shoulders, elbows, knees, etc.) and rotation axes according to the anatomical structure of the 3D character. Ensure that the joint positions are consistent with the key point positions in the captured data. Establish the parent-child hierarchical relationship of the bones to ensure the correct transmission of movements. For example, the movement of the thigh bone will be transmitted to the calf and foot. Determine the degrees of freedom (DOF) of the bones and limit the rotation range of the joints to conform to physiological laws.

[0052] Adjust the bone lengths according to the body shape (such as size and proportion) of the 3D character to match the target model. Use an automatic Retargeting algorithm to scale the action proportions in the captured data to fit the bones of the character. Bind the cleaned action data to the bone model to generate a preliminary animation. Utilize Retargeting technology to map the preprocessed action data to the joints of the character's bones to ensure the spatial consistency of the action trajectories. Automatically adjust for differences in bone proportions between different characters to ensure natural movements. Set the weight distribution for the bone binding of the character model to optimize the natural deformation of the character's skin. The weights at the joint parts need to be evenly distributed to avoid excessive stretching or deformation. Use an automatic skin weight calculation tool or manual fine-tuning to achieve higher precision.

[0053] S3: Optimize the bone animation using artificial intelligence, including action smoothing and key frame extraction. At the same time, generate the character's facial animation to achieve a natural match between expressions and actions, and complete the un-captured parts of the action through deep learning to generate smooth transition animations.

[0054] AI optimization of bone animation to eliminate minor jitters or discontinuities in the motion capture data and make the bone animation smoother. Use deep learning models (such as RNN or Transformer) to perform temporal analysis on the action data and predict the smooth transition values for each frame. Apply Bezier curves or Kalman filtering techniques to smooth and optimize the joint trajectories to ensure that the actions conform to the natural movement laws of the human body.

[0055] Key frame extraction is to reduce the redundant data in the bone animation. By extracting key frames, the core content of the action can be efficiently described. Use AI algorithms (such as principal component analysis PCA) to analyze the changes in key points in the action data and automatically select key frames. When extracting frames, give priority to retaining the peak points and direction change points of the action, and at the same time delete the intermediate redundant frames to reduce the computational complexity.

[0056] Generate and optimize facial animations to generate facial animations that naturally match the bone actions and enhance the emotional expressiveness of the character. Use expression recognition algorithms (such as convolutional neural network-based expression classification models) to identify and classify the facial motion capture data, and extract basic expression data (such as happy, angry, surprised, etc.). Map the facial capture data to the character's facial bones or Blend Shape control points to ensure the naturalness and smoothness of the expression changes.

[0057] Dynamic expression generation, generating subtle facial expressions that are not captured through AI to make the animation more realistic. Use generative adversarial networks (GANs) or variational autoencoders (VAEs) to predict detailed expressions (such as microexpressions, upward curling of the corners of the mouth, etc.). Adjust the weight distribution of facial bones and skin meshes in real time to optimize the naturalness of facial expressions and avoid unnecessary stretching or distortion.

[0058] Deep learning to complete uncaptured actions, predicting the uncaptured action trajectories through AI to generate complete animation data. Use deep learning models (such as LSTM or Transformer) to model time-series action data and predict the occluded or uncaptured joint trajectories. The model inputs include the captured joint positions, velocities, and direction changes, and the output is the completed full action trajectory.

[0059] Generate natural transition animations between action segments to make the action transitions smooth. Generate transition frames between action segments through a temporal generation model (such as Temporal GAN). Optimize joint accelerations and inertial changes to ensure that the transition animations conform to physical laws and have a natural visual effect.

[0060] S4: For multi-actor interaction scenarios, analyze the abnormal changes in the marker occlusion rate and marker mis-matching rate during the marker tracking process of the capture system, and evaluate the action tracking ability of the capture system for multiple characters.

[0061] In multi-actor interaction scenarios, use multi-view motion capture devices to record the marker positions, trajectories, and time-series data of each actor. Ensure that the capture system simultaneously collects the visibility status of all markers, including information on occluded markers.

[0062] Divide the captured marker data into three categories: Visible markers: Markers successfully tracked and recorded by the capture system. Occluded markers: Markers that cannot be captured due to actor action overlap or perspective limitations. Mis-matched markers: Markers that are incorrectly assigned to other actors or body parts.

[0063] Occlusion rate = number of occluded markers / total number of markers; Calculate the occlusion rate in each frame and record the average occlusion rate and peak occlusion rate of the entire capture process. Plot the change curve of the occlusion rate through time series to analyze the abnormal fluctuation points of the occlusion rate in multi-character interaction. Determine the specific action scenarios where high occlusion rates occur (such as hugs, wrestling, etc.) and evaluate the capture ability of the system in such high-occlusion situations.

[0064] Mismatch rate = Number of mismatched marker points / Total number of marker points; Count the number of mismatched marker points in each frame, and calculate the average mismatch rate and peak mismatch rate. Identify the specific positions and trajectories of the mismatched marker points, and analyze whether they are caused by action overlap, dense marker point distribution, or algorithm errors. For scenarios with a high mismatch rate, verify whether there are problems with the marker point allocation algorithm of the capture system (such as allocation rules based on distance or trajectory prediction).

[0065] After analyzing the abnormal changes in the marker point occlusion rate during the marker point tracking process of the capture system, generate a marker point occlusion rate fluctuation index. The method for obtaining the marker point occlusion rate fluctuation index is as follows:

[0066] Construct a time series of the marker point occlusion rate , where represents the marker point occlusion rate of the Nth frame, and the value range is 0 ≤ ≤ 10, N is the total number of frames in the time series, and set the length of the sliding window to W, that is, the number of frames included in the window. The larger the window, the higher the smoothing degree, but it may mask short-term fluctuations. For the tth frame, the sliding average is calculated as follows: ; When t < W, the number of frames included in the window is t (only calculate the existing data if it is less than W). is the occlusion rate of the ith frame. For the tth frame, the occlusion rate and the sliding average The absolute deviation calculation formula is: ; is the deviation value of the tth frame, indicating the difference between the current occlusion rate and the sliding average trend. Calculate the marker point occlusion rate fluctuation index, and the expression is: ; In the formula, CHK is the marker point occlusion rate fluctuation index, and N is the total number of frames, representing the length of the time series of the occlusion rate data.

[0067] When the marker point occlusion rate fluctuation index is large, it indicates that there are drastic changes in the marker point occlusion rate of the capture system, manifested as an abnormally high marker point occlusion rate in some frames, while it may be lower in other frames. This drastic fluctuation usually indicates that the tracking ability of the system in a multi-role interaction scenario is not stable enough and is easily affected by action overlap, occlusion, or perspective changes. This means that the capture system lacks robustness in complex scenarios, which may lead to incomplete or mis-mapped action trajectories of some characters, thus affecting the quality of the final animation.

[0068] When the marker point occlusion rate fluctuation index is small, it indicates that the occlusion rate of the capture system is relatively stable throughout the time series, with a small fluctuation range. This shows that the system can better handle the occlusion problem in multi-role interaction, and the tracking ability of the marker points is relatively stable. Even in the case of complex character actions and frequent overlaps, the capture system can still continuously track each marker point and generate complete and reliable action data. A capture system with a small fluctuation index usually has strong adaptability and accuracy, and is suitable for high-precision animation production and complex scene capture.

[0069] After analyzing the abnormal change of the marker point mis-matching rate during the marker point tracking process of the capture system, a marker point mis-matching rate abnormal index is generated. The acquisition method of the marker point mis-matching rate abnormal index is as follows:

[0070] Input data: Time series of marker point mis-matching rate , where represents the mis-matching rate of the G-th frame, calculate the mean of the sequence and subtract the mean from each point to obtain a zero-mean sequence ; s is the number of frames; perform FFT transformation on the mis-matching rate sequence to obtain the frequency domain representation ; ; In the formula, is the complex representation of the frequency component k, including amplitude and phase information, is the kernel function of the Fourier transform; for each frequency component , calculate the amplitude: ; In the formula, is the amplitude of the frequency component k, and are 's real and imaginary parts;

[0071] Determine the high-frequency range , the high-frequency part usually represents the fast-changing characteristics of the mis-matching rate. Let the high-frequency range be the last 30% of the total frequency or apply a specific frequency threshold (such as a frequency > 1 Hz); extract the amplitude set of the high-frequency components ; According to the amplitude of the high-frequency components, calculate the marker point mis-matching rate abnormal index, and the expression is: ; In the formula, FGH is the marker point mis-matching rate abnormal index.

[0072] When the abnormal index of the marker point mismatch rate is large, it indicates that the marker point allocation accuracy of the capture system in the multi-role interaction scenario is low, and the mismatch phenomenon is frequent and fluctuates violently. This may mean that the system lacks sufficient robustness in complex scenarios and cannot effectively distinguish the marker points of different roles or body parts, especially when actions overlap or switch rapidly, it performs particularly unstably. As a result, the generated action data may be inaccurate, with role actions crossing or misaligning, thus affecting the quality and usability of animation modeling.

[0073] When the abnormal index of the marker point mismatch rate is small, it shows that the capture system's allocation of marker points is relatively stable, with fewer mismatch phenomena and smaller fluctuations. This reflects that the system has a strong ability to track and allocate marker points in the multi-role interaction scenario, can accurately distinguish different roles and body parts, and can ensure the integrity and accuracy of the captured data even in complex action or frequent role interaction situations. Such a system is suitable for high-precision animation production and complex scene modeling, and the generated animation data is more reliable.

[0074] Convert the marker point occlusion rate fluctuation index and the marker point mismatch rate abnormal index into a comprehensive feature vector, use the comprehensive feature vector as the input of the machine learning model, and use the machine learning model to predict the action tracking accuracy value label of the capture system for multi-role as the prediction target, and use minimizing the sum of the prediction errors of all action tracking accuracy value labels of the capture system for multi-role as the training target to train the machine learning model until the sum of the prediction errors reaches convergence and then stop the model training, and determine the action tracking accuracy value of the capture system for multi-role according to the model output result, where the machine learning model is a polynomial regression model.

[0075] The method for obtaining the action tracking accuracy value of the capture system for multi-role is: obtain the corresponding function expression from the comprehensive feature vector training data of the trained machine learning model: ; where is the output function of the model, CHK is the marker point occlusion rate fluctuation index, FGH is the marker point mismatch rate abnormal index, is the action tracking accuracy value of the capture system for multi-role.

[0076] Compare the action tracking accuracy value of the capture system for multiple characters obtained with the pre-set action tracking accuracy reference threshold based on historical data. If the action tracking accuracy value of the capture system for multiple characters is greater than or equal to the pre-set action tracking accuracy reference threshold, it indicates that the capture system has high action tracking accuracy for multiple characters. At this time, generate an action tracking accurate signal, that is, the capture system has strong action tracking ability for multiple characters. If the action tracking accuracy value of the capture system for multiple characters is less than the pre-set action tracking accuracy reference threshold, it indicates that the capture system has low action tracking accuracy for multiple characters. At this time, generate an action tracking inaccurate signal, that is, the capture system has weak action tracking ability for multiple characters.

[0077] S5: When the capture system can accurately distinguish and record the independent action trajectories of each actor, simulate and test the generated 3D animation, discover and correct potential problems, and optimize the animation data in combination with the physics engine to make it conform to physical laws.

[0078] When the capture system can accurately distinguish and record the independent action trajectories of each actor, that is, the action tracking accuracy value of the capture system for multiple characters generated within a fixed time period is greater than or equal to the pre-set action tracking accuracy reference threshold, simulate and test the generated 3D animation, collect the action tracking accuracy values greater than or equal to the pre-set action tracking accuracy reference threshold generated in subsequent fixed time periods, establish a data set, calculate the mean and standard deviation of the data set, and after analyzing it, predict potential problems in advance according to the analysis results.

[0079] If the mean value of the action tracking accuracy values in the data set is greater than or equal to the reference threshold of the mean value of the action tracking accuracy values, and the standard deviation of the action tracking accuracy values is less than the reference threshold of the standard deviation of the action tracking accuracy values, it indicates that the system has high action tracking ability within a fixed time period, and at the same time shows good stability and consistency. The capture system can reliably distinguish and record the independent action trajectories of each character. The generated 3D animation has small errors, conforms to physical laws, and the subsequent optimization cost is low. Continue to maintain or slightly optimize the capture device and algorithm in the current configuration, and focus on subsequent animation physical enhancement and detail optimization.

[0080] If the mean value of the action tracking accuracy is greater than or equal to the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the action tracking accuracy, it indicates that there are still fluctuations in the system during some time periods, and the action tracking ability is not stable enough. Although the overall performance meets the standard, inaccurate tracking or large errors may occur in local time periods, affecting the local quality of the 3D animation. Short-term marker mis-matching or occlusion problems may occur in complex interaction scenarios. Focus on investigating the time periods with large errors (sources of high standard deviation), and optimize the robustness and anti-occlusion ability of the capture algorithm. Enhance the adaptability to dynamically changing scenarios (such as rapid character interaction).

[0081] If the mean value of the action tracking accuracy is less than the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the action tracking accuracy, it indicates that the overall performance of the capture system is insufficient and shows significant instability at the same time. The system has serious defects in the action tracking ability of multiple characters, and marker loss or mis-matching problems may occur frequently. There are many errors in the generated 3D animation, making it difficult to meet the physical laws and application requirements. Conduct a comprehensive optimization from the device configuration (such as increasing the viewing angle or improving the resolution) and the algorithm level (such as enhancing multi-target tracking). Perform frame-by-frame analysis on the time periods with large error fluctuations to find the key factors causing the system performance fluctuations.

[0082] If the mean value of the action tracking accuracy is less than the reference threshold of the mean value of the action tracking accuracy, and the standard deviation of the action tracking accuracy is less than the reference threshold of the standard deviation of the action tracking accuracy, it indicates that the system performance is poor but stable, and the error range is relatively consistent. When the system captures the actions of multiple characters, there may be systematic biases or insufficient capabilities, but the volatility is low. The generated animation has low quality and good overall consistency, but it cannot meet the high-precision requirements. Conduct structural improvements for systematic problems (such as insufficient marker resolution or poor occlusion handling). Optimize the basic capabilities of the device or algorithm to improve the accuracy of action tracking.

[0083] S6: Export the optimized animation in a compatible format and integrate it into the target scene. Complete the integration of the character and the scene through real-time rendering technology, add lights, materials, and special effects to generate the final high-quality animation effect.

[0084] Select an appropriate file format according to the target application requirements: FBX (Filmbox): Supports skeletal animation, materials, lights, and scene data, and is suitable for game engines (such as Unity, Unreal Engine) and film and television production. OBJ: Only exports static 3D models and is suitable for static scenes or further modeling processing. Alembic (ABC): Efficiently stores complex animation data (such as cloth or hair simulation) and is suitable for film special effects production.

[0085] Ensure the complete export of animation data: Skeletal data: including joint positions, hierarchical relationships, and key animation frames. Materials and textures: Bind materials and UV textures to avoid material loss after importing the model into the target software. Special effect information: such as particle effects or dynamic simulations (cloth, hair, etc.). Compress or simplify redundant data (such as redundant key frames or high-resolution textures) to improve the performance of the exported file and ensure compatibility and loading efficiency.

[0086] Import the exported animation file into the target scene editing tool (such as Unity, Unreal Engine, or Maya). Check whether the imported animation is completely loaded, especially the binding relationship and smoothness of the skeletal animation. Set the initial position and action path of the character in the scene: Ensure that the character animation dynamically matches the scene elements (such as terrain, obstacles) to avoid penetration phenomena. Introduce collision detection to ensure that the character's actions are consistent with the scene's physical rules.

[0087] Add and adjust the types and parameters of lights in the scene: Point light: used to simulate local light sources (such as torches, light bulbs). Directional light: simulates natural light (such as sunlight). Ambient light: evenly illuminates the scene to ensure the visibility of the character's details. Adjust the lighting effects: Introduce dynamic shadows and global illumination techniques to enhance the realism and visual depth of the scene.

[0088] Apply physically based rendering materials (PBR): Set parameters such as reflectivity, roughness, and metallicity according to the needs of the character and the scene to enhance realism. Add dynamic texture maps: such as the skin details of the character, the water surface reflection in the scene, and the skybox effect. Add particle effects: Character dynamic performance (such as dust effects when running, the trajectory of waving weapons). Scene effects (such as raindrops, flames, smoke). Dynamic simulation effects: such as cloth swaying, hair fluttering, calculated in real time through the physics engine.

[0089] Enable real-time rendering technologies (such as Ray Tracing or rasterization): Dynamically calculate the lighting in real time to achieve realistic light reflection, refraction, and scattering effects. Optimize the frame rate to ensure smooth playback of high-quality animations on the target device. Add post-processing effects: Depth of field: Highlights the foreground character and blurs the background details. Motion blur: Simulates the visual blur effect when the character moves quickly. Color correction: Adjusts the hue, brightness, and contrast of the animation to unify the visual style.

[0090] Output high-quality videos or interactive animation files according to the target platform: Video formats: such as MP4, MOV, for promotional, demonstration, etc. purposes. Interactive formats: such as EXE, HTML5, for games or virtual reality scenarios.

[0091] Test the animation effect on the target platform: Ensure that the actions of the character and the scene are coordinated, without problems such as model penetration and lag. Verify whether the lighting effects and special effects meet the expectations. Further optimize according to the feedback to ensure that the final quality of the animation meets the application requirements.

[0092] In this embodiment, a multi-view motion capture device is used to obtain the action data of the performer in real time. The position, rotation, and motion trajectory are recorded through marker points. After preprocessing the captured data, a skeletal model of the target 3D character is created, and the data is mapped onto the skeletal model. The basic animation is generated through scale adjustment and weight optimization. Subsequently, artificial intelligence technology is used to optimize the skeletal animation, including action smoothing, key frame extraction, and expression generation. Uncaptured actions are completed through deep learning to achieve smooth and natural transitional animations. For multi-actor interaction scenarios, the abnormal changes in the marker occlusion rate and false matching rate during the marker tracking of the capture system are analyzed to evaluate the multi-character tracking ability of the system. When the action tracking meets the accuracy requirements, the generated animation is simulated and tested to discover and correct potential problems, such as model penetration and joint overrun, and the animation data is optimized in combination with the physics engine to make it conform to the physical laws. Finally, the optimized animation is exported in a compatible format, integrated into the target scene, and lighting, materials, and special effects are added through real-time rendering technology to generate high-quality 3D animation effects.

[0093] Embodiment 2, please refer to Figure 2 As shown, the intelligent animation modeling system based on 3D stereo technology in this embodiment includes an action capture module, a data preprocessing and skeletal modeling module, an AI animation optimization module, a multi-character tracking evaluation module, an animation simulation and physical optimization module, and an animation rendering and integration module:

[0094] Action capture module: Use a multi-view motion capture device to obtain the action data of the performer in real time, including setting marker points at the key parts of the performer to record its position, rotation, and motion trajectory;

[0095] Data preprocessing and skeletal modeling module: After preprocessing the captured action data, create a skeletal model of the target 3D character, including joint positions and hierarchical relationships, and map the preprocessed action data onto the skeletal model through scale adjustment and weight optimization;

[0096] AI animation optimization module: Use artificial intelligence to optimize the skeletal animation, including action smoothing and key frame extraction, and at the same time generate the facial animation of the character to achieve natural matching of expressions and actions. Uncaptured parts of the actions are completed through deep learning to generate smooth transitional animations;

[0097] Multi-character tracking evaluation module: For multi-actor interaction scenarios, analyze the abnormal changes in the marker occlusion rate and marker false matching rate during the marker tracking process of the capture system to evaluate the action tracking ability of the capture system for multiple characters;

[0098] Animation Simulation and Physical Optimization Module: When the capture system can accurately distinguish and record the independent movement trajectories of each actor, simulate and test the generated 3D animation, discover and correct potential problems, including penetration and joint overrun, and optimize the animation data in combination with the physics engine to make it conform to physical laws;

[0099] Animation Rendering and Integration Module: Export the optimized animation in a compatible format and integrate it into the target scene. Complete the integration of the character and the scene through real-time rendering technology, add lights, materials, and special effects to generate the final high-quality animation effect.

[0100] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain a formula that is closest to the actual situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0101] It should be understood that the term "and / or" in this article is only a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Among them, A and B can be singular or plural. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood by referring to the context before and after.

[0102] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this article can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0103] As described above, this is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application.

Claims

1. An intelligent animation modeling method based on three-dimensional technology, characterized in that: The following steps are involved: S1: Use multi-view motion capture equipment to obtain the performer's motion data in real time, including setting markers on the performer's key parts to record their position, rotation and motion trajectory; S2: After preprocessing the captured motion data, a skeletal model of the target 3D character is created, including joint positions and hierarchical relationships, and the preprocessed motion data is mapped to the skeletal model through proportion adjustment and weight optimization; S3: Uses AI to optimize skeletal animation, including motion smoothing and keyframe extraction, while generating character facial animation to achieve a natural match between expression and motion, and uses deep learning to complete uncaptured motions to generate smooth transition animations; S4: For multi-actor interaction scenes, analyze the abnormal changes in the marker occlusion rate and marker mismatch rate during the marker tracking process of the capture system to evaluate the capture system's ability to track the movements of multiple characters; Specifically, it includes: analyzing the abnormal change of the marker point occlusion rate during the marker point tracking process of the capture system to generate a marker point occlusion rate fluctuation index. The method for obtaining the marker point occlusion rate fluctuation index is as follows: constructing a time series of the marker point occlusion rate , where represents the marker point occlusion rate of the Nth frame, and the value range is 0 ≤ ≤ 10, N is the total number of frames of the time series, and the length of the sliding window is set to W, that is, the number of frames included in the window. For the tth frame, the sliding average is calculated by the formula: ; when t < W, the number of frames included in the window is t, is the occlusion rate of the ith frame; for the tth frame, the occlusion rate and the sliding average The absolute deviation calculation formula is: ; is the deviation value of the tth frame, indicating the difference between the current occlusion rate and the sliding average trend. Calculate the marker point occlusion rate fluctuation index, and the expression is: ; in the formula, CHK is the marker point occlusion rate fluctuation index, and N is the total number of frames, representing the time series length of the occlusion rate data; S5: When the capture system can accurately distinguish and record the independent movement trajectory of each actor, the generated 3D animation is simulated and tested to find and correct potential problems, including model penetration and joint overrun, and the animation data is optimized in combination with the physics engine to make it conform to the laws of physics; S6: Export the optimized animation in a compatible format and integrate it into the target scene. Use real-time rendering technology to integrate the characters and scenes, add lighting, materials and special effects, and generate the final high-quality animation effect.

2. The intelligent animation modeling method based on three-dimensional technology according to claim 1, characterized in that: In S2, the bone length is adjusted according to the body shape of the three-dimensional character to match the target model. Specifically, the automatic Retargeting algorithm is used to scale the action ratio in the captured data to match the skeleton of the character, and the cleaned action data is bound to the skeleton model to generate a preliminary animation. The Retargeting technology is used to correspond the pre-processed action data to the joints of the character skeleton, and the differences in bone proportions of different characters are automatically adjusted. The weight distribution is set for the skeleton binding of the character model, and the weights of the joints need to be evenly distributed.

3. The intelligent animation modeling method based on three-dimensional technology according to claim 1 is characterized in that: In S4, the abnormal change of the mark point mismatch rate in the capture system during the mark point tracking process is analyzed to generate a mark point mismatch rate abnormal index. The method for obtaining the mark point mismatch rate abnormal index is as follows: construct a time series of the mark point mismatch rate ,in Represents the mismatch rate of the Gth frame, calculates the mean of the sequence and subtracts the mean from each point to obtain a zero-mean sequence ; s is the number of frames, the sequence of false matching rate Perform FFT transformation to obtain frequency domain representation ; ; In the formula, is the complex representation of the frequency component k, containing amplitude and phase information , is the kernel function of Fourier transform; For each frequency component, calculate the amplitude: ; In the formula, is the amplitude of frequency component k, and is the real and imaginary part of ; determine the high frequency range , extract the amplitude set of high-frequency components ; According to the amplitude of the high-frequency component, the abnormal index of the mismatch rate of the marker point is calculated, and the expression is: ; Where FGH is the abnormal index of the mismatch rate of the marked points.

4. The intelligent animation modeling method based on three-dimensional technology according to claim 3 is characterized in that: The marker point occlusion rate fluctuation index and the marker point mismatch rate anomaly index are converted into a comprehensive feature vector, and the comprehensive feature vector is used as the input of the machine learning model. The machine learning model uses each group of comprehensive feature vectors to predict the capture system's motion tracking accuracy value labels for multiple characters as the prediction target, and takes minimizing the sum of prediction errors of all capture systems' motion tracking accuracy value labels for multiple characters as the training target. The machine learning model is trained until the sum of prediction errors converges, and the model training is stopped. The motion tracking accuracy value of the capture system for multiple characters is determined according to the model output results, wherein the machine learning model is a polynomial regression model.

5. The intelligent animation modeling method based on three-dimensional technology according to claim 4 is characterized in that: The obtained motion tracking accuracy value of the capture system for multiple characters is compared with the motion tracking accuracy reference threshold value pre-set according to historical data. If the motion tracking accuracy value of the capture system for multiple characters is greater than or equal to the pre-set motion tracking accuracy reference threshold value, it means that the motion tracking accuracy of the capture system for multiple characters is high. At this time, a motion tracking accuracy signal is generated, that is, the capture system has a strong motion tracking capability for multiple characters; If the capture system's motion tracking accuracy value for multiple characters is less than a preset motion tracking accuracy reference threshold, it means that the capture system's motion tracking accuracy for multiple characters is low. At this time, an inaccurate motion tracking signal is generated, which means that the capture system's motion tracking capability for multiple characters is weak.

6. The intelligent animation modeling method based on three-dimensional technology according to claim 5 is characterized in that: In S5, when the capture system is able to accurately distinguish and record the independent motion trajectory of each actor, that is, the motion tracking accuracy value of the capture system for multiple characters generated within a fixed time period is greater than or equal to a preset motion tracking accuracy reference threshold, the generated three-dimensional animation is simulated and tested, and the motion tracking accuracy values ​​generated within a subsequent fixed time period that are greater than or equal to the preset motion tracking accuracy reference threshold are collected, and a data set is established, the mean and standard deviation of the data set are calculated, and after analyzing the data set, potential problems are predicted in advance based on the analysis results.

7. The intelligent animation modeling method based on three-dimensional technology according to claim 6 is characterized in that: If the mean value of the motion tracking accuracy values ​​in the data set is greater than or equal to the reference threshold of the mean value of the motion tracking accuracy values, and the standard deviation of the motion tracking accuracy values ​​is less than the reference threshold of the standard deviation of the motion tracking accuracy values, it indicates that the capture system can accurately distinguish and record the independent motion trajectory of each character, and the generated three-dimensional animation has a small error and conforms to the laws of physics. The capture device and algorithm continue to be maintained under the current configuration; If the mean value of the motion tracking accuracy is greater than or equal to the reference threshold of the mean value of the motion tracking accuracy, and the standard deviation of the motion tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the motion tracking accuracy, it indicates that the capture system still fluctuates in some time periods, and the motion tracking capability is unstable, which affects the local quality of the 3D animation. It is necessary to optimize the robustness and anti-occlusion capability of the capture algorithm to enhance its adaptability to dynamically changing scenes; If the mean value of the motion tracking accuracy is less than the reference threshold of the mean value of the motion tracking accuracy, and the standard deviation of the motion tracking accuracy is greater than or equal to the reference threshold of the standard deviation of the motion tracking accuracy, it indicates that the overall performance of the capture system is insufficient and unstable. The capture system has serious defects in its ability to track the motion of multiple characters, and comprehensive optimization is required from the device configuration and algorithm levels. If the mean value of the motion tracking accuracy value is less than the reference threshold of the mean value of the motion tracking accuracy value, and the standard deviation of the motion tracking accuracy value is less than the reference threshold of the standard deviation of the motion tracking accuracy value, it indicates that the performance of the capture system is poor but stable, but cannot meet the high-precision requirements. The equipment and algorithm need to be optimized to improve the accuracy of motion tracking.

8. An intelligent animation modeling system based on three-dimensional technology, used to implement the intelligent animation modeling method based on three-dimensional technology according to any one of claims 1 to 7, characterized in that: Including motion capture module, data preprocessing and skeleton modeling module, AI animation optimization module, multi-role tracking evaluation module, animation simulation and physical optimization module, and animation rendering and integration module: Motion capture module: Use multi-view motion capture equipment to obtain the performer's motion data in real time, including setting markers on the performer's key parts to record their position, rotation and movement trajectory; Data preprocessing and skeleton modeling module: After preprocessing the captured motion data, a skeleton model of the target 3D character is created, including joint positions and hierarchical relationships, and the preprocessed motion data is mapped to the skeleton model through proportion adjustment and weight optimization; AI Animation Optimization Module: Uses artificial intelligence to optimize skeletal animation, including motion smoothing and keyframe extraction, while generating character facial animation to achieve a natural match between expression and motion, and uses deep learning to complete uncaptured motions to generate smooth transition animations; Multi-character tracking evaluation module: For multi-actor interaction scenes, analyze the abnormal changes in the marker occlusion rate and marker mismatch rate during the marker tracking process of the capture system to evaluate the capture system's ability to track the movements of multiple characters; Animation simulation and physics optimization module: When the capture system can accurately distinguish and record the independent movement trajectory of each actor, the generated 3D animation is simulated and tested to find and correct potential problems, including penetration and joint overrun, and the animation data is optimized in combination with the physics engine to make it conform to the laws of physics; Animation rendering and integration module: Export the optimized animation in a compatible format and integrate it into the target scene. Use real-time rendering technology to integrate characters and scenes, add lighting, materials and special effects, and generate the final high-quality animation effect.

Images