Movie and television animation dynamic scene display optimization method based on deep learning

Abstract

The invention discloses a movie and television animation dynamic scene display optimization method based on deep learning. The method comprises the following steps: S1, obtaining a dynamic scene data set from a dynamic scene of movie and television animation; s2, preprocessing the dynamic scene data set; s3, inputting the preprocessed initial data set into a variational auto-encoder for training, and generating a diversified and vivid dynamic scene; s4, inputting the generated dynamic scene data and background dynamic element change data in the initial data set into a time sequence convolutional network model for time sequence processing; s5, performing synchronous optimization on a plurality of dynamic elements in the movie and television animation; s6, keeping smooth and coherent when role actions change quickly or scenes are switched frequently; s7, dynamically adjusting display parameters in the scene by using the trained variational auto-encoder and the time sequence convolutional network model; and S8, outputting the dynamic scene display after real-time optimization. According to the method, the fluency of role actions, the natural transition of ambient light and shadow and the time sequence consistency of background dynamic elements are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of film and television animation, and in particular to a method for optimizing the display of dynamic scenes in film and television animation based on deep learning. Background Art

[0002] In the prior art, the optimization of dynamic scene display in film and television animation mainly relies on manual adjustment and traditional rendering technology. During the production process of film and television animation, the changes in character movements, environmental lighting and background elements in the dynamic scene display often require manual and detailed adjustments. Whenever the character movements, lighting changes or background elements are updated, the designer needs to ensure the continuity and consistency of the scene display effect through repeated rendering and debugging. Although the traditional method can guarantee the quality of the display effect to a certain extent, its efficiency is extremely low and requires a lot of manpower investment. In complex scenes, manual adjustment can easily lead to visual incoherence and scene distortion problems. With the complexity and diversification of film and television animation scenes, traditional manual optimization methods are difficult to respond to complex dynamic changes in real time, resulting in a display effect that does not meet the audience's expectations.

[0003] In the prior art, some scene display optimization methods use preset animation scripts and fixed rendering pipelines to generate animation effects by adjusting preset parameters. This technical approach is too single and rigid when facing complex scenes and cannot effectively handle issues such as the smoothness of character movements, the natural transition of light and shadow, and the synchronous optimization of multiple dynamic elements with timing consistency of background elements. Especially in fast-switching scenes, traditional technologies often cannot respond quickly to sudden changes in the scene, which can easily lead to delays or distortions in the display effects, affecting the audience's visual experience.

[0004] In addition, the existing technology lacks coordination between multiple elements in dynamic scenes. The character actions, object movements, light and shadow changes, and background dynamic elements in dynamic scenes need to be coordinated with each other to ensure the overall consistency of the scene. Existing technical means usually process these dynamic elements separately. The lack of an effective synchronization optimization mechanism can easily lead to visual incoherence between the elements, affecting the final display effect.

[0005] To sum up, the existing technology has the following main shortcomings: first, the optimization of dynamic scene display relies on manual adjustment and traditional rendering methods, which is inefficient and easily leads to display distortion; second, the existing technology lacks synchronous optimization processing of multiple dynamic elements, and cannot effectively solve the coordination problems between character movements, light and shadow changes, and background dynamic elements; finally, the existing technology does not respond in time when facing complex scene changes, and it is difficult to ensure the real-time and continuity of the display effect. Summary of the invention

[0006] One purpose of the present invention is to propose a method for optimizing the display of dynamic scenes in film and television animations based on deep learning. The present invention significantly improves the fluency of character movements, the natural transition of environmental light and shadow, and the timing consistency of background dynamic elements.

[0007] A method for optimizing the display of dynamic scenes in film and television animations based on deep learning according to an embodiment of the present invention comprises the following steps:

[0008] S1. Obtain dynamic scene datasets from dynamic scenes of film and television animations, including 3D models, character actions, environmental light and shadow changes, and background dynamic element changes in dynamic scenes;

[0009] S2. Preprocessing the dynamic scene data set, including gridding the 3D model data, extracting motion vectors from the character action data, separating features from the ambient light and shadow change data, and extracting time series information from the background dynamic element change data, to generate an initial data set for training the variational autoencoder and the time series convolutional network;

[0010] S3, inputting the preprocessed initial data set into the variational autoencoder for training, using the variational autoencoder to encode and decode the three-dimensional model data, character action data, and environmental light and shadow change data in the dynamic scene, to generate diversified and realistic dynamic scenes;

[0011] S4, inputting the generated dynamic scene data and the background dynamic element change data in the initial data set into the temporal convolutional network model for temporal processing, and the temporal convolutional network model optimizes the continuity of the role action, the object movement trajectory and the environmental light and shadow changes in the time dimension according to the temporal information of the dynamic scene;

[0012] S5. By combining the scene data generated by the variational autoencoder with the time series data optimized by the temporal convolutional network, multiple dynamic elements in the film and television animation are synchronously optimized, including the smoothness of the character's movements, the natural transition of the ambient light and shadow, and the temporal consistency of the background dynamic elements, so that the visual effects between the various elements in the dynamic scene are consistent and coordinated;

[0013] S6. Input the jointly optimized dynamic scene data into the rendering engine, and adjust the display parameters in real time to optimize the dynamic scene display effect in the film and television animation, and maintain smoothness and coherence when the character's actions change rapidly or the scenes switch frequently;

[0014] S7. During the animation playback process, the dynamic scene is monitored in real time, and the display parameters in the scene are dynamically adjusted using the trained variational autoencoder and temporal convolutional network model. When the character moves, the light and shadow change, or the background elements change, the rendering effect is re-optimized in real time to ensure that any scene changes maintain the best display effect;

[0015] S8. Output dynamic scene display that has been optimized in real time. The dynamic scene display has smooth time evolution, coherent character movements, natural light and shadow changes, and consistent background dynamic element display.

[0016] Optionally, the specific method of S1 includes:

[0017] S11, obtaining the geometric structure and topological relationship of all objects in the scene, and generating a complete three-dimensional model data set M3 by capturing and modeling the three-dimensional coordinates of the geometric structure and topological relationship of each object;

[0018] S12. Obtaining character action dataset A r , the character action data set includes the motion trajectory, posture changes and joint movements of the character in the scene, and generates the motion vector of the character action by capturing and tracking the key points of the character;

[0019] S13. Obtaining the environmental light and shadow change data set L e The environmental light and shadow change data set includes the dynamic changes of the position of the light source in the scene, the light intensity, the shadow projection and the light reflection, and generates the characteristic data of the environmental light and shadow change by real-time capture of the illumination model in the scene;

[0020] S14. Obtaining background dynamic element change data set E b The background dynamic element change data set includes the motion trajectory, deformation and appearance change of the background objects in the scene, and generates the time series data of the background dynamic elements by tracking the background elements and extracting the dynamic change features;

[0021] S15, integrating the three-dimensional model dataset, the character action dataset, the environmental light and shadow change dataset, and the background dynamic element change dataset to form a dynamic scene dataset:

[0022] D={M3,A r ,L e ,E b}.

[0023] Optionally, the specific method of S2 includes:

[0024] S21, gridding and segmenting the 3D model data set M3, by performing a 3D space gridding operation on the geometric structure of the model, to segment the 3D model data into a plurality of small grid data, and the geometric attribute of each small grid is G i , and obtain the segmented 3D grid dataset M 3,g :

[0025] M 3,g ={G1,G2,...,G n};

[0026] S22. Character action dataset A r Perform motion vector extraction, extract the key frames of each character's action, and calculate the motion trajectory vector V between each key frame a , and obtain the motion vector dataset V of the character's action a,r :

[0027] V a,r = {V a,1 ,V a,2 ,...,V a,n};

[0028] S23, environmental light and shadow change data set L e Perform feature separation, including separation of light source position, light intensity and shadow projection light and shadow features, and generate light and shadow feature dataset F l , and obtain the light and shadow change feature dataset F l,e :

[0029] F l,e ={F l,1 ,F l,2 ,...,F l,n};

[0030] S24, background dynamic element change data set E b Extract time series information and generate a time series feature dataset T by performing time series analysis on the motion trajectory and deformation of background elements e , and obtain the time series data set T of the background dynamic elements e,b :

[0031] T e,b ={T e,1 ,T e,2 ,...,T e,n};

[0032] S25, integrating the segmented 3D mesh dataset, the character action motion vector dataset, the light and shadow change feature dataset, and the background dynamic element time series dataset to generate an initial dataset D for training the variational autoencoder and the time series convolutional network init :

[0033] D init ={M 3,g ,V a,r ,F l,e ,T e,b}.

[0034] Optionally, the S3 specific method includes:

[0035] S31, the preprocessed initial data set Dinit Input a hierarchical variational autoencoder model for training. The hierarchical variational autoencoder model includes a multi-level latent space encoder and decoder structure, which generates hierarchical latent space variables by independently encoding different levels in the dynamic scene.

[0036] S32, for the three-dimensional model data set M 3,g Perform the first level of latent space encoding to generate the latent space variable z corresponding to the low-dimensional features of the three-dimensional model dataset M,1 :

[0037]

[0038] in, Indicates the rate of change of 3D model data over time, Represents the gradient change of the 3D model in the spatial dimension, combined with the velocity field v to capture the dynamic changes of the model geometry, λ1·R M is the regularization term for the geometric constraints of the model, Ω is the spatial domain of the three-dimensional model data;

[0039] S33, character action dataset V a,r Perform the second level of latent space encoding to generate the latent space variable z corresponding to the temporal dynamic characteristics of the character action A,2 :

[0040]

[0041] in, The time difference of each key frame captured for the character's action changes over time, J(V a,r ) represents the acceleration Jacobian matrix of the action, which is used to calculate the acceleration motion characteristics of the character. α1 is the weight factor, balancing the influence of acceleration and position. i ·M i Represents the motion information of the character's joints, q i Joint position, M i is the corresponding mass matrix, β i is the weight factor, T is the time domain;

[0042] S34, environmental light and shadow change data set F l,e Perform the third level of latent space encoding to generate latent space variables z corresponding to the illumination, shadow and reflection characteristics of the ambient light and shadow changes L,3 :

[0043]

[0044] Among them, L s N represents the light direction L in the scene sThe dot product with the surface normal N captures the angular changes of light and shadow. It represents the rate of change of light intensity over time. The light and shadow follow the speed v of the light l The dynamic gradient of the light intensity is γ, C shadow is the correction term for scene shadow changes, δ is the shadow weight adjustment factor, and S is the spatial domain of light and shadow changes;

[0045] S35, through the dynamic management layer to the different levels of latent space variables z L,1 ,z L,2 ,z L,3 Perform synchronous management and decoding. The dynamic management layer dynamically decodes the latent space variables according to different scene change requirements and controls the decoding order and frequency of each level:

[0046] M′ 3,g =Decoder1(z M,1 );

[0047] V′ a,r =Decoder2(z A,2 );

[0048] F′ l,e =Decoder3(z L,3 );

[0049] Among them, M' 3,g 、V' a,r 、F' l,e They are respectively the decoded 3D model data, character action data and environmental light and shadow change data, representing the optimized dynamic scene data.

[0050] Optionally, the specific method of S4 includes:

[0051] S41, the generated dynamic scene data M' 3,g 、V' a,r 、F' l,e The background dynamic element change data set T' e,b Input the temporal convolutional network model together for temporal processing;

[0052] S42, temporal convolutional network model for character action data V' a,r Optimize the continuity of the time dimension and use the extended causal convolution of the temporal convolutional network model to capture the temporal relationship of the character's actions at different times to generate the character's action data after the time dimension is optimized.

[0053] S43, temporal convolutional network model for object motion trajectory data M' 3,gPerform continuous optimization to capture the motion patterns of objects in dynamic scenes and generate object motion trajectory data after time dimension optimization

[0054] S44, Temporal Convolutional Network Model for Environmental Light and Shadow Change Data F' l,e Optimize the time dimension, capture the time series characteristics of light and shadow changes, and generate optimized environmental light and shadow change data

[0055] S45, background dynamic element change data T' e,b Optimize the continuity of the time dimension, capture the timing characteristics of background dynamic elements in the scene, and generate optimized background dynamic element data

[0056] Optionally, the specific method of S5 includes:

[0057] S51, the scene data M' generated by the variational self-encoder 3,g 、V' a,r 、F' l,e Optimizing Time Series Data with Temporal Convolutional Networks Perform synchronous optimization, jointly encode and decode scene data and time series data in multi-level latent space, and generate a synchronously optimized multi-element dynamic scene dataset;

[0058] S52. Optimize the smoothness of the character's movements by smoothing the trajectory of the character's movements in the time dimension to maintain motion continuity between consecutive key frames:

[0059]

[0060] in, To synchronize the optimized character action data, is the time rate of change of the character's action, is the Jacobian matrix of the character action acceleration, which adjusts the balance between acceleration and position. α2 is the weight coefficient for adjusting the influence of acceleration. M i is the mass matrix, capturing the kinematic mass of the character's joints, q i is the joint position vector, β i is the weight factor of the mass matrix;

[0061] S53. Optimize the natural transition of ambient light and shadow, and combine the light and shadow changes in the time series with the lighting angle in the scene to make the light and shadow transition natural and without mutation:

[0062]

[0063] in, To synchronize the optimized ambient light and shadow data, and They represent the time difference of light and shadow changes in the time dimension, γ1 is the light intensity adjustment factor, δ1 is the shadow weight adjustment factor, and L is the light and shadow change domain;

[0064] S55. Optimize the timing consistency of background dynamic elements to ensure that the movement trajectory of background elements is consistent with the character's actions and light and shadow changes in timing:

[0065]

[0066] in, To synchronize the optimized background dynamic element data, is the time change rate of the background dynamic element, d j ·v j Represents the product of the displacement direction and velocity of the background element, combined with the weight factor η j Adjust the motion trajectory of background elements, is the temporal transformation matrix of the background element, which adjusts the temporal consistency of the element, ξ is the deformation correction factor, and B is the background element change domain;

[0067] S56. Synchronize the optimized character action data Ambient lighting data Dynamic element data with background Integration to generate the final optimized dynamic scene display dataset D final :

[0068]

[0069] Among them, D final This is the final optimized dynamic scene display dataset. α3 and α4 are the weight adjustment parameters for character actions and environmental light and shadow, balancing the impact of different data types in scene optimization. W1 and W2 are the multi-dimensional weight matrices for character actions and light and shadow changes, respectively. It is the temporal transformation matrix of the scene model, capturing the temporal changes of objects and background elements in dynamic scenes.

[0070] The beneficial effects of the present invention are:

[0071] (1) The present invention introduces a multi-level variational autoencoder, which generates diversified and realistic dynamic scenes by independently encoding and decoding three-dimensional model data, character action data, and environmental light and shadow change data. Through the latent space optimization of the multi-level variational autoencoder, it can adaptively generate a variety of complex scenes, thereby improving the diversity of animation scenes and the realism of visual effects. In complex dynamic scenes, the multi-level variational autoencoder can automatically learn and reconstruct the detailed features of each element to ensure the realism and natural transition of the scene display.

[0072] (2) The present invention uses a temporal convolutional network to optimize the continuity of character actions, object motion trajectories, and environmental light and shadow changes in film and television animations in the temporal dimension. The extended causal convolution mechanism of the temporal convolutional network can capture the temporal relationship between multiple dynamic elements in the scene, ensuring that the display of character actions, light and shadow changes, and background dynamic elements at different time points is consistent and coherent, solving the problem of incoherent display caused by the lack of synchronization of multiple dynamic elements in traditional methods.

[0073] (3) The present invention combines the scene data generated by the variational autoencoder with the time series data optimized by the temporal convolutional network, and adopts a deep fusion model to synchronously optimize multiple dynamic elements, so that the character movements, environmental lighting and background dynamic elements have consistency and coordination in visual effects. Through the joint optimization mechanism, the coordinated optimization of multi-element display is achieved, ensuring that the visual performance of different elements in the scene is highly consistent and coherent, avoiding visual abruptness and incoherence in the display process, thereby greatly improving the overall expressiveness of the animation scene and the audience's sense of immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0075] Figure 1 A flowchart of a method for optimizing the display of dynamic scenes in film and television animations based on deep learning proposed by the present invention;

[0076] Figure 2 This is a structural schematic diagram of the encoding and decoding of dynamic scene data by a hierarchical variational autoencoder model in a deep learning-based film and television animation dynamic scene display optimization method proposed in the present invention. DETAILED DESCRIPTION

[0077] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0078] refer to Figure 1-2, a method for optimizing the display of dynamic scenes in film and television animation based on deep learning, comprising the following steps:

[0079] S1. Obtain dynamic scene datasets from dynamic scenes of film and television animations, including 3D models, character actions, environmental light and shadow changes, and background dynamic element changes in dynamic scenes;

[0080] S2. Preprocessing the dynamic scene data set, including gridding the 3D model data, extracting motion vectors from the character action data, separating features from the ambient light and shadow change data, and extracting time series information from the background dynamic element change data, to generate an initial data set for training the variational autoencoder and the time series convolutional network;

[0081] S3, inputting the preprocessed initial data set into the variational autoencoder for training, using the variational autoencoder to encode and decode the three-dimensional model data, character action data, and environmental light and shadow change data in the dynamic scene, to generate diversified and realistic dynamic scenes;

[0082] S4, inputting the generated dynamic scene data and the background dynamic element change data in the initial data set into the temporal convolutional network model for temporal processing, and the temporal convolutional network model optimizes the continuity of the role action, the object movement trajectory and the environmental light and shadow changes in the time dimension according to the temporal information of the dynamic scene;

[0083] S5. By combining the scene data generated by the variational autoencoder with the time series data optimized by the temporal convolutional network, multiple dynamic elements in the film and television animation are synchronously optimized, including the smoothness of the character's movements, the natural transition of the ambient light and shadow, and the temporal consistency of the background dynamic elements, so that the visual effects between the various elements in the dynamic scene are consistent and coordinated;

[0084] S6. Input the jointly optimized dynamic scene data into the rendering engine, and adjust the display parameters in real time to optimize the dynamic scene display effect in the film and television animation, and maintain smoothness and coherence when the character's actions change rapidly or the scenes switch frequently;

[0085] S7. During the animation playback process, the dynamic scene is monitored in real time, and the display parameters in the scene are dynamically adjusted using the trained variational autoencoder and temporal convolutional network model. When the character moves, the light and shadow change, or the background elements change, the rendering effect is re-optimized in real time to ensure that any scene changes maintain the best display effect;

[0086] S8. Output dynamic scene display that has been optimized in real time. The dynamic scene display has smooth time evolution, coherent character movements, natural light and shadow changes, and consistent background dynamic element display.

[0087] In this implementation, the specific method of S1 includes:

[0088] S11, obtaining the geometric structure and topological relationship of all objects in the scene, and generating a complete three-dimensional model data set M3 by capturing and modeling the three-dimensional coordinates of the geometric structure and topological relationship of each object;

[0089] S12. Obtaining character action dataset A r ,The character action dataset includes the motion trajectory, posture changes and joint movements of the ,character in the scene. The motion vector of the character ,action is generated by capturing and motion tracking the key points of the ,character;

[0090] S13. Obtaining the environmental light and shadow change data set L e ,The environmental light and shadow change dataset includes the dynamic changes of the light source position, light intensity, shadow projection and light reflection in the scene. ,The characteristic data of environmental light and shadow changes are generated by real-time ,capturing the illumination model in the scene;

[0091] S14. Obtaining background dynamic element change data set E b ,The background dynamic element change dataset includes the motion trajectory, deformation and appearance change of the background objects in the scene. ,The time series data of the background dynamic elements are generated by tracking and ,extracting the dynamic change features of the background elements;

[0092] S15, integrating the three-dimensional model dataset, the character action dataset, the environmental light and shadow change dataset, and the background dynamic element change dataset to form a dynamic scene dataset:

[0093] D={M3,A r ,L e ,E b}.

[0094] In this implementation, the specific method of S2 includes:

[0095] S21, gridding and segmenting the 3D model data set M3, by performing a 3D space gridding operation on the geometric structure of the model, to segment the 3D model data into a plurality of small grid data, and the geometric attribute of each small grid is G i , and obtain the segmented 3D grid dataset M 3,g :

[0096] M 3,g ={G1,G2,...,G n};

[0097] S22. Character action dataset A r Perform motion vector extraction, extract the key frames of each character's action, and calculate the motion trajectory vector V between each key frame a, and obtain the motion vector dataset V of the character's action a,r :

[0098] V a,r = {V a,1 ,V a,2 ,...,V a,n};

[0099] S23, environmental light and shadow change data set L e Perform feature separation, including separation of light source position, light intensity and shadow projection light and shadow features, and generate light and shadow feature dataset F l , and obtain the light and shadow change feature dataset F l,e :

[0100] F l,e ={F l,1 ,F l,2 ,...,F l,n};

[0101] S24, background dynamic element change data set E b Extract time series information and generate a time series feature dataset T by performing time series analysis on the motion trajectory and deformation of background elements e , and obtain the time series data set T of the background dynamic elements e,b :

[0102] T e,b ={T e,1 ,T e,2 ,...,T e,n};

[0103] S25, integrating the segmented 3D mesh dataset, the character action motion vector dataset, the light and shadow change feature dataset, and the background dynamic element time series dataset to generate an initial dataset D for training the variational autoencoder and the time series convolutional network init :

[0104] D init ={M 3,g ,V a,r ,F l,e ,T e,b}.

[0105] In this implementation, the specific method of S3 includes:

[0106] S31, the preprocessed initial data set D init Input a hierarchical variational autoencoder model for training. The hierarchical variational autoencoder model includes a multi-level latent space encoder and decoder structure, which generates hierarchical latent space variables by independently encoding different levels in the dynamic scene.

[0107] S32, for the three-dimensional model data set M 3,g Perform the first level of latent space encoding to generate the latent space variable z corresponding to the low-dimensional features of the three-dimensional model dataset M,1 :

[0108]

[0109] in, Indicates the rate of change of 3D model data over time, Represents the gradient change of the 3D model in the spatial dimension, combined with the velocity field v to capture the dynamic changes of the model geometry, λ1·R M is the regularization term for the geometric constraints of the model, Ω is the spatial domain of the three-dimensional model data;

[0110] S33, character action dataset V a,r Perform the second level of latent space encoding to generate the latent space variable z corresponding to the temporal dynamic characteristics of the character action A,2 :

[0111]

[0112] in, The time difference of each key frame captured for the character's action changes over time, J(V a,r ) represents the acceleration Jacobian matrix of the action, which is used to calculate the acceleration motion characteristics of the character. α1 is the weight factor, balancing the influence of acceleration and position. i ·M i Represents the motion information of the character's joints, q i Joint position, M i is the corresponding mass matrix, β i is the weight factor, T is the time domain;

[0113] S34, environmental light and shadow change data set F l,e Perform the third level of latent space encoding to generate latent space variables z corresponding to the illumination, shadow and reflection characteristics of the ambient light and shadow changes L,3 :

[0114]

[0115] Among them, L s N represents the light direction L in the scene s The dot product with the surface normal N captures the angular changes of light and shadow. It represents the rate of change of light intensity over time. The light and shadow follow the speed v of the light l The dynamic gradient of the light intensity is γ, C shadowis the correction term for scene shadow changes, δ is the shadow weight adjustment factor, and S is the spatial domain of light and shadow changes;

[0116] S35, through the dynamic management layer to the different levels of latent space variables z L,1 ,z L,2 ,z L,3 Perform synchronous management and decoding. The dynamic management layer dynamically decodes the latent space variables according to different scene change requirements and controls the decoding order and frequency of each level:

[0117] M′ 3,g =Decoder1(z M,1 );

[0118] V′ a,r =Decoder2(z A,2 );

[0119] F′ l,e =Decoder3(z L,3 );

[0120] Among them, M' 3,g 、V' a,r , F' l,e They are respectively the decoded 3D model data, character action data and environmental light and shadow change data, representing the optimized dynamic scene data.

[0121] In this implementation manner, the specific method of S4 includes:

[0122] S41, the generated dynamic scene data M' 3,g 、V' a,r , F' l,e The background dynamic element change data set T' e,b Input the temporal convolutional network model together for temporal processing;

[0123] S42, temporal convolutional network model for character action data V' a,r Optimize the continuity of the time dimension and use the extended causal convolution of the temporal convolutional network model to capture the temporal relationship of the character's actions at different times to generate the character's action data after the time dimension is optimized.

[0124] S43, temporal convolutional network model for object motion trajectory data M' 3,g Perform continuous optimization to capture the motion patterns of objects in dynamic scenes and generate object motion trajectory data after time dimension optimization

[0125] S44, Temporal Convolutional Network Model for Environmental Light and Shadow Change Data F' l,eOptimize the time dimension, capture the time series characteristics of light and shadow changes, and generate optimized environmental light and shadow change data

[0126] S45, background dynamic element change data T' e,b Optimize the continuity of the time dimension, capture the timing characteristics of background dynamic elements in the scene, and generate optimized background dynamic element data

[0127] In this implementation manner, the specific method of S5 includes:

[0128] S51, the scene data M' generated by the variational self-encoder 3,g 、V' a,r , F' l,e Optimizing Time Series Data with Temporal Convolutional Networks Perform synchronous optimization, jointly encode and decode scene data and time series data in multi-level latent space, and generate a synchronously optimized multi-element dynamic scene dataset;

[0129] S52. Optimize the smoothness of the character's movements by smoothing the trajectory of the character's movements in the time dimension to maintain motion continuity between consecutive key frames:

[0130]

[0131] in, To synchronize the optimized character action data, is the time rate of change of the character's action, is the Jacobian matrix of the character action acceleration, which adjusts the balance between acceleration and position. α2 is the weight coefficient for adjusting the influence of acceleration. M i is the mass matrix, capturing the kinematic mass of the character's joints, q i is the joint position vector, β i is the weight factor of the mass matrix;

[0132] S53. Optimize the natural transition of ambient light and shadow, and combine the light and shadow changes in the time series with the lighting angle in the scene to make the light and shadow transition natural and without mutation:

[0133]

[0134] in, To synchronize the optimized ambient light and shadow data, and They represent the time difference of light and shadow changes in the time dimension, γ1 is the light intensity adjustment factor, δ1 is the shadow weight adjustment factor, and L is the light and shadow change domain;

[0135] S55. Optimize the timing consistency of background dynamic elements to ensure that the movement trajectory of background elements is consistent with the character's actions and light and shadow changes in timing:

[0136]

[0137] in, To synchronize the optimized background dynamic element data, is the time change rate of the background dynamic element, d j ·v j Represents the product of the displacement direction and velocity of the background element, combined with the weight factor η j Adjust the motion trajectory of background elements, is the temporal transformation matrix of the background element, which adjusts the temporal consistency of the element, ξ is the deformation correction factor, and B is the background element change domain;

[0138] S56. Synchronize the optimized character action data Ambient lighting data Dynamic element data with background Integration to generate the final optimized dynamic scene display dataset D final :

[0139]

[0140] Among them, D final This is the final optimized dynamic scene display dataset. α3 and α4 are the weight adjustment parameters for character actions and environmental light and shadow, balancing the impact of different data types in scene optimization. W1 and W2 are the multi-dimensional weight matrices for character actions and light and shadow changes, respectively. It is the temporal transformation matrix of the scene model, capturing the temporal changes of objects and background elements in dynamic scenes.

[0141] Embodiment 1:

[0142] In order to verify the feasibility and effectiveness of the present invention in optimizing the display of dynamic scenes in film and television animations, the experimenters selected the publicly available animation datasets RenderMan and MovieStitch for experiments. The RenderMan dataset contains a variety of complex animation scenes, including battle scenes, dialogue scenes, and environmental change scenes. The data covers character movements, light and shadow changes, and dynamic changes of background elements. The MovieStitch dataset mainly includes multiple continuous scene transitions, involving rapid switching of different time points and perspectives. The data editing time is from January 2021 to March 2022. The dataset provides a good basis for the experimenters to verify the dynamic scene display optimization method of the present invention.

[0143] In the experiment, the experimenter divided the two data sets into training sets and test sets in a ratio of 8:2. In order to effectively optimize the multiple elements of the dynamic scene, the experimenter first preprocessed the data of the character action, three-dimensional scene and light and shadow changes. Specifically, the three-dimensional model was encoded using a variational autoencoder to generate diverse scene data, and the time series characteristics of the character action and environmental light and shadow changes were captured through a temporal convolutional network. The model was trained using the Adam optimizer, with the initial learning rate set to 0.001. The training was performed for 150 epochs, and the mini-batch was set to 32. In order to ensure the accuracy and display effect of the model, the experimenter compared the traditional manual adjustment method with the automatic optimization method of the present invention.

[0144] During the experiment, the experimenters mainly used three evaluation indicators to verify the display effect: display coherence, motion smoothness and light and shadow consistency. Display coherence mainly measures the visual consistency of different elements in the scene, motion smoothness evaluates the naturalness of the character's transition between different key frames, and light and shadow consistency evaluates whether the light and shadow changes are smooth and without mutations.

[0145] In the RenderMan dataset, the experimenter selected a scene with high-speed character movement, frequent light and shadow changes, and dynamic changes in background elements, with a duration of 60 seconds. The traditional manual adjustment method and the method of the present invention were used for optimization. The experimental data showed that the traditional method took about 72 hours to process the scene due to its reliance on manual frame-by-frame adjustment, and the generated display effect had obvious light and shadow mutations and incoherent movements. The specific results are shown in Table 1 below:

[0146] Table 1 Experimental results on the RenderMan dataset

[0147]

[0148] As can be seen from Table 1, the present invention is significantly superior to the traditional method in terms of display continuity, and the action fluency and light and shadow consistency are also significantly improved. The maximum time difference is controlled within 0.05 seconds, ensuring the natural transition of the character and the smooth change of light and shadow. The traditional method relies on manual adjustment, consumes a lot of time, and the display effect is poor.

[0149] In the MovieStitch dataset, the experimenter selected a clip containing multiple fast-switching scenes for testing. The characters in the scene frequently switch between different perspectives, the light and shadow change dramatically, and the background elements are complex. The experimental results show that when dealing with fast scene switching, the traditional method is prone to the problem of action and background being out of sync, and the light and shadow transition is not natural enough. On the contrary, the method of the present invention can quickly adapt to scene switching and ensure the timing consistency of action, light and shadow, and background elements. The specific results are shown in Table 2 below:

[0150] Table 2 Experimental results on the MovieStitch dataset

[0151]

[0152]

[0153] The data in Table 2 further demonstrates the advantages of the present invention in processing complex scene switching, especially in the synchronization of action and background elements. The maximum time difference is controlled within 0.06 seconds, which is much better than the 0.38 seconds of the traditional method. In addition, the present invention is nearly 5 times more efficient in processing time than the traditional method.

[0154] When training the above dataset, the experimenters introduced multimodal feature extraction technology, used the pre-trained ResNet network to extract the visual features in the scene, and used the pre-trained BERT model to embed the text information in the scene description. Through the multimodal fusion of visual and text features, the experimenters further improved the model's ability to understand the details of the scene. In the present invention, the dimension of the visual features is 1024, and the dimension of the text features is 768. A 2D convolutional layer is used to align the dimensions of the visual features and the text features and then input them into the deep learning model for training.

[0155] In order to deal with the problem of unbalanced sample data, the experimenters introduced a weighted loss function during the training process and made appropriate adjustments to the minority class samples to ensure that the model can effectively optimize various dynamic elements in complex scenes. In addition, the experimenters also used reinforcement learning technology to enable the model to respond quickly when the dynamic scene changes, and to optimize the character movements, light and shadow changes, and background elements in real time, avoiding the visual lag when switching scenes in traditional methods.

[0156] Through experimental comparison, the experimenter verified the superiority of the present invention in the display of dynamic scenes in film and television animation. The traditional method is time-consuming and prone to problems of scene incoherence and sudden changes in light and shadow. The present invention significantly improves the naturalness and coherence of the display effect through the joint optimization of the variational autoencoder and the temporal convolutional network, especially in complex scenes. In terms of processing efficiency, the method of the present invention is more than 5 times faster than the traditional manual adjustment method, and the display effect also significantly improves the audience's visual experience.

[0157] In summary, the present invention provides an efficient and accurate automatic optimization method in the dynamic display of complex film and television animation scenes, which successfully solves the problems of low efficiency and poor display effect in traditional methods. Through rich experimental data, the experimenters have proved that the present invention not only has high practicality, but also has significant advantages in complex scene processing, and can provide strong technical support for the efficient production of film and television animation.

[0158] The present invention introduces a multi-level variational autoencoder, which generates diversified and realistic dynamic scenes by independently encoding and decoding three-dimensional model data, character action data, and environmental light and shadow change data. Through the latent space optimization of the multi-level variational autoencoder, it can adaptively generate a variety of complex scenes, thereby improving the diversity of animation scenes and the realism of visual effects. In complex dynamic scenes, the multi-level variational autoencoder can automatically learn and reconstruct the detailed features of each element to ensure the realism and natural transition of the scene display.

[0159] The present invention uses a temporal convolutional network to optimize the continuity of character actions, object motion trajectories, and environmental light and shadow changes in film and television animations in the time dimension. The extended causal convolution mechanism of the temporal convolutional network can capture the temporal relationship between multiple dynamic elements in the scene, ensuring that the display of character actions, light and shadow changes, and background dynamic elements at different time points is consistent and coherent, solving the problem of incoherent display caused by the lack of synchronization of multiple dynamic elements in traditional methods.

[0160] The present invention combines the scene data generated by the variational autoencoder with the time series data optimized by the temporal convolutional network, and adopts a deep fusion model to synchronously optimize multiple dynamic elements, so that the character movements, environmental light and shadow, and background dynamic elements have consistency and coordination in visual effects. Through the joint optimization mechanism, the coordinated optimization of multi-element display is achieved, ensuring that the visual performance of different elements in the scene is highly consistent and coherent, avoiding visual abruptness and incoherence in the display process, thereby greatly improving the overall expressiveness of the animation scene and the audience's sense of immersion.

[0161] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for optimizing the display of dynamic scenes in film and television animation based on deep learning, characterized in that: The steps include: S1. Obtain dynamic scene datasets from dynamic scenes of film and television animations, including 3D models, character actions, environmental light and shadow changes, and background dynamic element changes in dynamic scenes; S2. Preprocess the dynamic scene dataset to generate an initial dataset for training the variational autoencoder and temporal convolutional network; S3, inputting the preprocessed initial data set into the variational autoencoder for training, using the variational autoencoder to encode and decode the three-dimensional model data, character action data, and environmental light and shadow change data in the dynamic scene, to generate diversified and realistic dynamic scenes; S4, inputting the generated dynamic scene data and the background dynamic element change data in the initial data set into the temporal convolutional network model for temporal processing; S5. Synchronously optimize multiple dynamic elements in film and television animation by combining scene data generated by variational autoencoders with time series data optimized by temporal convolutional networks; S6. Input the jointly optimized dynamic scene data into the rendering engine, and adjust the display parameters in real time to optimize the dynamic scene display effect in the film and television animation, and maintain smoothness and coherence when the character's actions change rapidly or the scenes switch frequently; S7. During the animation playback process, the dynamic scene is monitored in real time, and the display parameters in the scene are dynamically adjusted using the trained variational autoencoder and temporal convolutional network model. When the character moves, the light and shadow change, or the background elements change, the rendering effect is re-optimized in real time; S8. Output dynamic scene display that has been optimized in real time.

2. The method for optimizing the display of dynamic scenes in film and television animation based on deep learning according to claim 1, characterized in that: The specific method of S1 includes: S11, obtaining the geometric structure and topological relationship of all objects in the scene, and generating a complete three-dimensional model data set M3 by capturing and modeling the three-dimensional coordinates of the geometric structure and topological relationship of each object; S12. Obtaining character action dataset A r , the character action data set includes the motion trajectory, posture changes and joint movements of the character in the scene, and generates the motion vector of the character action by capturing and tracking the key points of the character; S13. Obtaining the environmental light and shadow change data set L e The environmental light and shadow change data set includes the dynamic changes of the position of the light source in the scene, the light intensity, the shadow projection and the light reflection, and generates the characteristic data of the environmental light and shadow change by real-time capture of the illumination model in the scene; S14. Obtaining background dynamic element change data set E b The background dynamic element change data set includes the motion trajectory, deformation and appearance change of the background objects in the scene, and generates the time series data of the background dynamic elements by tracking the background elements and extracting the dynamic change features; S15, integrating the three-dimensional model dataset, the character action dataset, the environmental light and shadow change dataset, and the background dynamic element change dataset to form a dynamic scene dataset: D={M3,A r ,L e ,HAVE BEEN b }。 3. The method for optimizing the display of dynamic scenes in film and television animation based on deep learning according to claim 1, characterized in that: The S2 specific method includes: S21, meshing and segmenting the 3D model data set M3, by performing a 3D space meshing operation on the geometric structure of the model, to segment the 3D model data into a plurality of small mesh data, and the geometric attribute of each small mesh is G i , and obtain the segmented 3D grid dataset M 3,g : M 3,g ={G1,G2,...,G n }; S22. Character action dataset A r Perform motion vector extraction, extract the key frames of each character's action, and calculate the motion trajectory vector V between each key frame a , and obtain the motion vector dataset V of the character's action a,r : V a,r ={V a,1 ,V a,2 ,...,V a,n }; S23, environmental light and shadow change data set L e Perform feature separation, including separation of light source position, light intensity and shadow projection light and shadow features, and generate light and shadow feature dataset F l , and obtain the light and shadow change feature dataset F l,e : F l,e ={F l,1 ,F l,2 ,...,F l,n }; S24, background dynamic element change data set E b Extract time series information and generate a time series feature dataset T by performing time series analysis on the motion trajectory and deformation of background elements e , and obtain the time series data set T of the background dynamic elements e,b : T e,b ={T e,1 ,T e,2 ,...,T e,n }; S25, integrating the segmented 3D mesh dataset, the character action motion vector dataset, the light and shadow change feature dataset, and the background dynamic element time series dataset to generate an initial dataset D for training the variational autoencoder and the time series convolutional network init : D init ={M 3,g ,V a,r ,F l,e ,T e,b }。 4. The method for optimizing the display of dynamic scenes in film and television animation based on deep learning according to claim 1, characterized in that: The S3 specific method includes: S31, the preprocessed initial data set D init Input a hierarchical variational autoencoder model for training. The hierarchical variational autoencoder model includes a multi-level latent space encoder and decoder structure, which generates hierarchical latent space variables by independently encoding different levels in the dynamic scene. S32, for the three-dimensional model data set M 3,g Perform the first level of latent space encoding to generate the latent space variable z corresponding to the low-dimensional features of the three-dimensional model dataset M,1 : in, Indicates the rate of change of 3D model data over time, Represents the gradient change of the 3D model in the spatial dimension, combined with the velocity field v to capture the dynamic changes of the model geometry, λ1·R M is the regularization term for the geometric constraints of the model, Ω is the spatial domain of the three-dimensional model data; S33, character action dataset V a,r Perform the second level of latent space encoding to generate the latent space variable z corresponding to the temporal dynamic characteristics of the character action A,2 : in, The time difference of each key frame captured for the character's action changes over time, J(V a,r ) represents the acceleration Jacobian matrix of the action, which is used to calculate the acceleration motion characteristics of the character. α1 is the weight factor, balancing the influence of acceleration and position. i ·M i Represents the motion information of the character's joints, q i Joint position, M i is the corresponding mass matrix, β i is the weight factor, T is the time domain; S34, environmental light and shadow change data set F l,e Perform the third level of latent space encoding to generate latent space variables z corresponding to the illumination, shadow and reflection characteristics of the ambient light and shadow changes L,3 : Among them, L s N represents the light direction L in the scene s The dot product with the surface normal N captures the angular changes of light and shadow. It represents the rate of change of light intensity over time. The light and shadow follow the speed v of the light l The dynamic gradient of the light intensity is γ, C shadow is the correction term for scene shadow changes, δ is the shadow weight adjustment factor, and S is the spatial domain of light and shadow changes; S35, through the dynamic management layer to the different levels of latent space variables z L,1 , z L,2 , z L,3 Perform synchronous management and decoding. The dynamic management layer dynamically decodes the latent space variables according to different scene change requirements and controls the decoding order and frequency of each level: M′ 3，g =Decoder1(z M,1 ); In a，r =Decoder2(from A,2 ); F′ l，e =Decoder3(from L,3 ); Among them, M′ 3，g , V′ a，r , F′ l，e They are respectively the decoded 3D model data, character action data and environmental light and shadow change data, representing the optimized dynamic scene data.

5. The method for optimizing the display of dynamic scenes in film and television animation based on deep learning according to claim 1, characterized in that: The S4 specific method includes: S41, generate dynamic scene data M' 3，g , V′ a，r , F′ l，e The background dynamic element change dataset T′ e，b Input the temporal convolutional network model together for temporal processing; S42, temporal convolutional network model for character action data V′ a，r Optimize the continuity of the time dimension and use the extended causal convolution of the temporal convolutional network model to capture the temporal relationship of the character's actions at different times to generate the character's action data after the time dimension is optimized. S43, temporal convolutional network model for object motion trajectory data M′ 3，g Perform continuous optimization to capture the motion patterns of objects in dynamic scenes and generate object motion trajectory data after time dimension optimization S44, Temporal Convolutional Network Model for Environmental Light and Shadow Change Data F′ l，e Optimize the time dimension, capture the time series characteristics of light and shadow changes, and generate optimized environmental light and shadow change data S45, background dynamic element change data T' e，b Optimize the continuity of the time dimension, capture the timing characteristics of background dynamic elements in the scene, and generate optimized background dynamic element data 6. The method for optimizing the display of dynamic scenes in film and television animation based on deep learning according to claim 1, characterized in that: The specific method of S5 includes: S51, the scene data M' generated by the variational self-encoder 3,g 、V' a,r 、F' l,e Optimizing Time Series Data with Temporal Convolutional Networks Perform synchronous optimization, jointly encode and decode scene data and time series data in multi-level latent space, and generate a synchronously optimized multi-element dynamic scene dataset; S52. Optimize the smoothness of the character's movements by smoothing the trajectory of the character's movements in the time dimension to maintain motion continuity between consecutive key frames: in, To synchronize the optimized character action data, is the time rate of change of the character's action, is the Jacobian matrix of the character action acceleration, which adjusts the balance between acceleration and position. α2 is the weight coefficient for adjusting the influence of acceleration. M i is the mass matrix, capturing the kinematic mass of the character's joints, q i is the joint position vector, β i is the weight factor of the mass matrix; S53. Optimize the natural transition of ambient light and shadow, and combine the light and shadow changes in the time series with the lighting angle in the scene to make the light and shadow transition natural and without mutation: in, To synchronize the optimized ambient light and shadow data, and They represent the time difference of light and shadow changes in the time dimension, γ1 is the light intensity adjustment factor, δ1 is the shadow weight adjustment factor, and L is the light and shadow change domain; S55. Optimize the timing consistency of background dynamic elements to ensure that the movement trajectory of background elements is consistent with the character's actions and light and shadow changes in timing: in, To synchronize the optimized background dynamic element data, is the time change rate of the background dynamic element, d j ·v j Represents the product of the displacement direction and velocity of the background element, combined with the weight factor η j Adjust the motion trajectory of background elements, is the temporal transformation matrix of the background element, which adjusts the temporal consistency of the element, ξ is the deformation correction factor, and B is the background element change domain; S56. Synchronize the optimized character action data Ambient lighting data Dynamic element data with background Integration to generate the final optimized dynamic scene display dataset D final : Among them, D final This is the final optimized dynamic scene display dataset. α3 and α4 are the weight adjustment parameters for character actions and environmental light and shadow, balancing the impact of different data types in scene optimization. W1 and W2 are the multi-dimensional weight matrices for character actions and light and shadow changes, respectively. It is the temporal transformation matrix of the scene model, capturing the temporal changes of objects and background elements in dynamic scenes.

Images