High-fidelity dynamic scene video generation method based on single static image

Abstract

The invention discloses a high-fidelity dynamic scene video generation method based on a single static image, and relates to the technical field of computer vision and video generation, and the method comprises the steps: obtaining key elements and potential dynamic information in an image through deep understanding and semantic deconstruction of a static scene; through dynamic representation and modeling, spatial-temporal feature decoupling and complex dynamic scene modeling are realized; according to the method, the dynamic complexity can be processed and the visual high fidelity can be ensured at the same time, the problems of dynamic incoherence, detail loss and the like when a video is generated from a single static image are solved, the method can be widely applied to the fields of film and television production, virtual reality and the like, and the method is suitable for popularization and application. According to the method, the rich details and the sense of reality of the original image can be reserved to the greatest extent while the complex dynamic state is generated, the space-time consistency is kept in the whole video sequence, the image quality is prevented from being sacrificed due to dynamic state generation, and the method has high use value.

Description

Technical Field

[0001] This invention relates to the field of computer vision and video generation technology, specifically to a method for generating high-fidelity dynamic scene videos based on a single static image. Background Art

[0002] With the rapid development of digital media technology, the demand for generating dynamic videos from static images is increasing, and it has broad application prospects in film and television production, virtual reality, game development, and other fields. Currently, existing methods for generating videos from single static images often struggle to simultaneously handle the complexity of dynamic scenes and the high fidelity of the generated videos. Some methods can only handle simple object translations or rotations, and cannot cope with complex dynamics involving scene-level elements, multiple elements, interactions, non-rigidity, and environmental linkages. Therefore, a high-fidelity dynamic scene video generation method based on single static images is needed.

[0003] A search revealed that Chinese application number CN202311865081.7 discloses a video generation method and apparatus based on static images. In the video generation method based on static images provided in this specification, a static target image is obtained, and the viewpoint change process required when generating video based on the static target image is determined. It is evident that the aforementioned patent applications have shortcomings: traditional methods are prone to losing details of the original image during the generation of dynamic content, such as texture blurring and geometric distortion, making it difficult to guarantee the high fidelity of the video and reducing the implementation quality of the method.

[0004] To address the aforementioned problems, this invention proposes a method for generating high-fidelity dynamic scene videos based on a single static image. The aim of this invention is to solve the problem in existing technologies where it is difficult to simultaneously handle complex dynamics and ensure high fidelity when generating videos from a single static image. Summary of the Invention

[0005] The purpose of this invention is to provide a high-fidelity dynamic scene video generation method based on a single static image. This method uses this device to solve the problem that traditional methods in the background technology are prone to loss of details in the original image during the dynamic generation process, such as texture blurring and geometric distortion, making it difficult to guarantee the high fidelity of the video and reducing the implementation quality of the method.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for generating high-fidelity dynamic scene video based on a single static image, comprising the following steps: Step 1: Static Scene Deep Understanding and Semantic Deconstruction: Deep feature extraction is performed on the input single static RGB image to identify and segment key elements in the image, and the potential motion patterns or change trends in the scene are inferred by combining prior knowledge base or physical model. Deep neural networks are used to extract multi-level deep features from a single static RGB image; key elements in the image are identified and segmented, including foreground objects, background environment, texture regions, lighting information, spatial layout, and semantic labels; based on scene semantics, combined with prior knowledge base or physical model, potential motion patterns or change trends in the scene are inferred. Step 2, Dynamic Representation and Modeling: Predict dense motion fields or optical flow in future frames at the pixel level or feature level, separate depth features into spatial and temporal features, and model the interactions between different elements in the scene as well as global and local dynamics. Predict dense motion fields or optical flow in future frames at the pixel or feature level to capture complex non-rigid deformations, occlusion relationships, viewpoint changes, and the interaction of environmental elements; clearly separate the depth features extracted in step one into spatial features and temporal features, where spatial features are responsible for maintaining high-fidelity details of static elements, and temporal features are responsible for driving dynamic evolution; understand the interactions between different elements in the scene and the global and local dynamics. Step 3: High-fidelity video synthesis: Using predicted motion field or temporal features to drive spatial features to deform and render frame by frame in the temporal dimension to generate a video sequence, preserving the original image details and handling complex environmental changes to ensure the temporal continuity of the video. By leveraging predicted motion field and temporal features, spatial features are driven to deform and render frame by frame in the temporal dimension to generate video sequences, ensuring smooth and natural transitions between frames; video details are preserved and enhanced, including retaining the fine texture of the original image during motion deformation, and maintaining high geometric consistency and realistic appearance of content from new perspectives when dynamically including viewpoint changes; complex environmental changes are handled, including background changes and scene-level events.

[0007] Preferably, the static scene deep understanding and semantic deconstruction steps include: A deep neural network is used to extract multi-level deep features from the input static image. The deep neural network includes CNN or ViT. Identify and segment foreground objects, background environment, texture regions, lighting information, and spatial layout in an image, and annotate with semantic labels; Based on semantic tags combined with prior knowledge bases or physical models, we infer potential motion patterns or trends in a scene.

[0008] Preferably, in the dynamic representation and modeling step, the spatial features and temporal features in the depth features are separated. The spatial features are used to preserve the texture, color, and shape details of static elements, while the temporal features are used to drive dynamic evolution.

[0009] Preferably, the temporal coherence generation in the high-fidelity video synthesis step is achieved by predicting motion field and temporal features, driving spatial features to deform and render frame by frame, ensuring smooth and natural transitions between frames, and avoiding flickering, jittering, or discontinuous motion.

[0010] Preferably, the detail preservation and enhancement in the high-fidelity video synthesis step includes: Texture preservation: During motion deformation, the fine textures in the original image are preserved and passed to subsequent frames; New perspective synthesis: When dynamics involve changes in perspective, the content under the new perspective maintains high geometric consistency and appearance realism.

[0011] Preferably, the complex environmental change processing in the high-fidelity video synthesis step includes: Background change processing: Processing cloud flow, water ripples, weather changes, and changes in light and shadow over time; Scene-level event handling: Simulates scene-level events such as flame combustion, smoke diffusion, particle effects, and plant growth.

[0012] Preferably, the high-fidelity video synthesis further includes a model architecture and a training strategy. The model architecture adopts a generative model, which can perform multi-scale and spatiotemporal joint discrimination, judging both the authenticity of a single frame and the coherence between frames.

[0013] Preferably, the generated model includes a motion prediction module, which employs an optical flow prediction network, a deformation field prediction network, or a Transformer-based temporal modeling module.

[0014] Preferably, the generated model also includes a feature fusion and rendering module, which uses deformable convolution, optical flow-based feature warping, or neural radiation field techniques to transform and render spatial features according to temporal features.

[0015] Compared with the prior art, the present invention has the following beneficial effects:

[0016] This application proposes a high-fidelity dynamic scene video generation method based on a single static image. This method can handle complex dynamics, going beyond simple object translation or rotation. It can handle complex dynamics at the scene level, involving multiple elements, interactions, non-rigidity, and environmental linkage. It ensures high fidelity by preserving the rich details and realism of the original image to the greatest extent while generating complex dynamics, and maintains spatiotemporal consistency throughout the video sequence, avoiding sacrificing image quality due to dynamic generation. It has a wide range of applications and can be widely used in film and television production, virtual reality, game development, and other fields, making it highly practical.

[0017] This application proposes a high-fidelity dynamic scene video generation method based on a single static image. The innovative integration of high fidelity and dynamic scene is reflected in the method. The core innovative value of this method lies in simultaneously overcoming two key challenges in the extremely difficult task of generating video from a single image: (1) Dynamic complexity: It goes beyond simple object translation or rotation and can handle complex dynamics such as scene-level, multi-element, interaction, non-rigid, and environmental linkage; (2) Visual quality: While generating complex dynamics, it retains the rich details and realism of the original image to the greatest extent and maintains spatiotemporal consistency throughout the video sequence, such as lighting, texture, and geometry, to achieve high fidelity standards and avoid sacrificing image quality due to dynamic generation. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the overall steps of the high-fidelity dynamic scene video generation method for a single static image according to the present invention. Detailed Implementation

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] To further understand the content of this invention, a detailed description of the invention will be provided in conjunction with the accompanying drawings.

[0021] Combination Figure 1 A method for generating high-fidelity dynamic scene videos based on a single static image, comprising the following steps: Step 1: In-depth understanding and semantic deconstruction of static scenes Deep feature extraction: Deep neural networks such as CNN (Convolutional Neural Network) or ViT (Visual Transformer) are used to perform multi-level deep feature extraction on the input single static RGB image in order to obtain the deep information of the image.

[0022] Scene analysis: The deep neural network described above identifies and segments key elements in the image, such as foreground objects (people, vehicles, etc.), background environment (sky, ground, etc.), texture regions, lighting information, spatial layout, and semantic tags, such as "river", "clouds", "crowds", etc.

[0023] Potential dynamic inference: Based on the identified scene semantics, combined with prior knowledge base (such as the known that rivers usually flow downstream) or physical model (such as fluid dynamics model used to infer water flow), the potential motion patterns or trends of change that may exist in the scene are inferred, such as the direction of water flow, the direction of cloud drift, the general movement trend of crowds, etc. This step is the basis for generating reasonable dynamics.

[0024] Step 2: Dynamic Representation and Modeling Dense motion field and future frame optical flow prediction: The motion prediction module (such as optical flow prediction network, deformation field prediction network or Transformer-based temporal modeling module) predicts pixel-level or feature-level dense motion fields or future frame optical flow, thereby capturing complex non-rigid deformations (such as the bending motion of the human body), occlusion relationships (such as one object occluding another object), viewpoint changes (such as lens rotation), and the linkage of environmental elements (such as the coordinated movement of leaves and grass in the wind).

[0025] Spatiotemporal feature decoupling: The depth features extracted in step 1 are explicitly separated into spatial features and temporal features. Spatial features include information such as appearance and structure, which are responsible for maintaining high-fidelity details of static elements, such as texture, color, and shape; temporal features include information such as motion and change, which are responsible for driving dynamic evolution.

[0026] Dynamic scene modeling: The model understands the interactions between different elements in the scene, such as object collisions, hydrodynamic effects (water flowing against objects), and changes in light and shadow as objects move. It also understands global and local dynamics, such as the camera movement of the entire scene and the independent movement of a certain object within the scene.

[0027] Step 3: High-fidelity video compositing Temporal coherence generation: Utilizing predicted motion field and temporal features, spatial features are driven to deform and render frame by frame in the temporal dimension to generate video sequences. Precise calculations ensure extremely smooth and natural transitions between frames, avoiding flickering, jitter, or motion discontinuities.

[0028] Preservation and enhancement of details: Texture Preservation: During motion deformation, a special texture transfer algorithm is used to accurately preserve and transfer the fine textures in the original image (such as skin pores, fabric textures, and leaf veins) to subsequent frames, avoiding blurring or distortion.

[0029] New perspective compositing: When dynamics involve changes in perspective (such as camera rotation or object flipping), new perspective compositing technology is used to maintain high geometric consistency and appearance realism of the content under the new perspective. For example, after an object is flipped, the texture and shape of its back remain consistent with the front.

[0030] Handling complex environmental changes: Background changes: For cloud flow, the movement trajectory and shape changes of the clouds are simulated; for water ripples, the propagation and undulation of water waves are simulated based on the principles of fluid dynamics; for weather changes, raindrops or snowflakes are generated through a particle system, and their falling trajectory and accumulation effect are simulated; for changes in light and shadow over time, the changes in the shadows of objects are calculated based on changes in the position of the sun or the movement of the light source.

[0031] Scene-level events: For flame combustion, the shape, color and spread of flames are simulated based on a combustion model; for smoke diffusion, a fluid dynamics model is used to simulate the flow and diffusion of smoke; for particle effects (raindrops, snowflakes), the generation, movement and disappearance of particles are controlled through a particle system; for plant growth simulation, the germination, growth and flowering processes of plants are simulated according to the laws of plant growth.

[0032] Step 4: Model Architecture and Training Strategy Generative Model: A multi-scale, spatiotemporal joint discriminative generative model is employed. This model can determine both the realism of a single frame (spatial fidelity) and the coherence between frames (temporal consistency). This discriminative approach ensures that the generated video achieves a high level of quality in both single frames and between frames.

[0033] Motion prediction module: Optical flow prediction network, deformation field prediction network or Transformer-based temporal modeling module can be used to accurately predict motion information.

[0034] Feature fusion and rendering: Using techniques such as deformable convolution, optical flow-based feature warping, and neural radiation fields, spatial features are efficiently and faithfully transformed and rendered according to temporal features to generate high-quality video frames.

[0035] Training Strategy: During training, a large number of image-video pairs containing dynamic scenes are used as training data. The model parameters are continuously optimized through backpropagation, enabling the model to learn the ability to generate high-fidelity dynamic videos from static images. Simultaneously, constraints on spatiotemporal consistency and detail preservation are emphasized during training to improve model performance. Example 1:

[0036] In the generation of dynamic videos of natural landscapes (river and vegetation scenes) This embodiment generates high-fidelity dynamic videos from static images of natural landscapes including rivers, trees, and grasslands. The specific steps are as follows: (1) Input an RGB image containing a river, trees on the bank, and grass. Use ResNet-50 (a type of CNN) to extract deep features and obtain features such as the outline of the river, the texture of the water flow, the distribution of tree branches and leaves, and the density of the grass in the image.

[0037] Scene analysis: The foreground elements of trees and grass, the background elements of sky, and the main element of river are segmented and labeled with semantic tags such as "river", "trees", "grass", and "sky".

[0038] Potential dynamic inference: Combining fluid dynamics models (prior knowledge), it is inferred that the direction of river flow is from left to right, and the flow velocity varies with the width of the river channel; based on meteorological common sense (prior knowledge base), it is inferred that the swaying direction of tree branches and grass under a light breeze is from right to left, and the swaying amplitude of branches and leaves is greater than that of grass.

[0039] (2) The motion prediction module uses an optical flow prediction network (such as PWC-Net) to predict the pixel-level dense motion field on the river surface, the non-rigid deformation optical flow of tree branches and leaves, and the overall tilt optical flow of grassland.

[0040] Spatiotemporal feature decoupling: Spatial features retain the ripple texture of the river, the bark texture and leaf veins of the trees, and the grass blade morphology; temporal features extract the motion vector of the water flow, the frequency and amplitude of the swaying of branches and leaves, and the rate of change of the tilt angle of the grassland over time.

[0041] Dynamic scene modeling: Simulate the slight erosion of the riverbank grassland by the flow of water, and the dynamic changes in the shadows of trees on the ground as the branches and leaves sway.

[0042] (3) High-fidelity video synthesis Temporal coherence generation: Based on predicted optical flow, the river features are deformed frame by frame (ripple propagation), the tree branches and leaves sway frame by frame, and the grassland tilts frame by frame to generate a 30-second video (30fps) with smooth and jitter-free inter-frame transitions.

[0043] Details are preserved: the ripples of the river are clearly conveyed in the flow, the veins of the leaves remain unblurred as they sway, and the tips of the grass blades remain sharp as they tilt.

[0044] Complex environmental changes: Add a few slowly moving clouds in the sky to simulate sunlight filtering through branches and leaves, creating light spots that move slowly across the ground over time. Example 2:

[0045] Dynamic video generation of characters and indoor scenes (dance movement simulation) This embodiment generates a high-fidelity video of continuous dance movements from a static image containing a single person dancing indoors. The steps are as follows: (1) Input an indoor RGB image (including floor, walls and mirror) of a person in the starting position of a dance, and use ViT to extract deep features to obtain the person's skeletal structure, clothing texture and indoor spatial layout.

[0046] Scene analysis: The foreground figures (labeled "human body"), the background floor (wood texture), the walls (wallpaper pattern), and the mirror (reflective area) are segmented.

[0047] Potential dynamic inference: Combine human kinematics model (prior knowledge) to infer the possible sequence of dance movements of the character, and infer the synchronous movement trend of the character's image in the mirror based on the principle of mirror reflection.

[0048] (2) The motion prediction module adopts a Transformer-based temporal modeling module to predict the motion trajectory of human joints, the non-rigid deformation of clothing, and the optical flow of the mirror reflection area.

[0049] Spatiotemporal feature decoupling: Spatial features retain skin pores, clothing knit textures, floor wood grain, and wallpaper patterns; temporal features extract the motion angles of human joints, the phase of clothing movement, and delay compensation for mirror reflection.

[0050] Dynamic scene modeling: Simulates the interaction between clothing and air when a character rotates, as well as the stretching and deformation of the character's shadow on the floor as the character moves.

[0051] (3) Temporal coherence generation: Drive the character features to deform frame by frame according to the joint movement trajectory to generate a 10-second dance video with smooth and uninterrupted movements, and the mirror reflection is strictly synchronized with the character's movements.

[0052] Detail preservation: the knit texture of clothing remains undistorted when stretched, skin pores remain clearly visible when the body rotates, and the wood grain of the floor maintains texture continuity under the shadows cast by the moving figure.

[0053] Complex environmental changes: Add the hands of a clock on the wall that turn slowly to simulate the gradual decrease in indoor brightness caused by the dimming of light outside the window. Example 3:

[0054] Dynamic video generation of weather changes (sunny turning to light rain scene) This embodiment generates a high-fidelity dynamic video that gradually changes from sunny to light rain from static images containing city streets. The steps are as follows: (1) Input a sunny city street RGB image (including buildings, road surface, pedestrians and sky), and use CNN+ViT hybrid network to extract the reflection of glass curtain walls of buildings, brick texture of road surface, details of pedestrian clothing and cloud distribution in the sky.

[0055] Scene analysis: The scene is segmented into buildings (foreground / background), road surface (background), pedestrians (foreground), and sky (distant view), and labeled with semantic tags such as "glass curtain wall", "brick road surface", "pedestrians", and "clouds".

[0056] Potential dynamic inference: Combining meteorological models to infer the gradual thickening of cloud layers and the falling trend of raindrops from nothing to something, and inferring the possible rain-avoidance actions of pedestrians based on common sense about pedestrian behavior.

[0057] (2) The motion prediction module uses an optical flow prediction network + particle system to predict the overall moving optical flow of clouds, the particle motion trajectory of raindrops, and the change in the moving speed of pedestrians.

[0058] Spatiotemporal feature decoupling: Spatial features retain the reflective texture of the glass curtain wall, the uneven texture of the brick pavement, and the patterns on pedestrians' clothing; temporal features extract the cloud accumulation speed, the density increase rate of raindrops, and the changes in pedestrian walking frequency.

[0059] Dynamic scene modeling: Simulates the splashing effect of raindrops falling on a glass curtain wall, the puddles formed by raindrops falling on the road, and the slight swaying of the umbrella when a pedestrian holds an umbrella.

[0060] (3) Temporal coherence generation: drive the clouds to gather frame by frame, increase the number of raindrop particles frame by frame, and accelerate the pedestrians frame by frame to generate a 40-second video with natural and abrupt weather transitions.

[0061] Details preserved: the reflection of the glass curtain wall gradually becomes blurred after being covered by raindrops, but the texture transitions naturally; the brick and stone patterns on the road surface are clearly presented through the reflection of the water after puddles are formed; and the patterns on pedestrians' clothing do not become distorted even when their limbs are deformed by walking quickly.

[0062] Complex environmental changes: Add building shadows that gradually fade as the cloud layer thickens, simulating the fine water droplets produced by raindrops hitting the road surface.

[0063] In summary, by taking a single static image (RGB image) as input, the method of this application can output a coherent video that demonstrates the dynamic changes implied or inferred in the scene of the input image. The core of this method is to reasonably infer and generate a physically believable, visually coherent, and detailed dynamic evolution process from the information of a single frame of the image. This invention can generate a scene video that conforms to physical laws, preserves details, and is dynamically coherent from a single static image, fully demonstrating the technical advantages of "high fidelity" and "complex dynamics".

[0064] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0065] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating high-fidelity dynamic scene video based on a single still image, characterized in that: Includes the following steps: Step 1: Static Scene Deep Understanding and Semantic Deconstruction: Deep feature extraction is performed on the input single static RGB image to identify and segment key elements in the image, and the potential motion patterns or change trends in the scene are inferred by combining prior knowledge base or physical model. Deep neural networks are used to extract multi-level deep features from a single input static RGB image. Identify and segment key elements in an image, including foreground objects, background environment, texture regions, lighting information, spatial layout, and semantic labels; based on scene semantics, combined with a prior knowledge base or physical model, infer potential motion patterns or trends in the scene. Step 2, Dynamic Representation and Modeling: Predict dense motion fields or optical flow in future frames at the pixel level or feature level, separate depth features into spatial and temporal features, and model the interactions between different elements in the scene as well as global and local dynamics. Predict dense motion fields or optical flow in future frames at the pixel or feature level to capture complex non-rigid deformations, occlusion relationships, viewpoint changes, and the interaction of environmental elements; clearly separate the depth features extracted in step one into spatial features and temporal features, where spatial features are responsible for maintaining high-fidelity details of static elements, and temporal features are responsible for driving dynamic evolution; understand the interactions between different elements in the scene and the global and local dynamics. Step 3: High-fidelity video synthesis: Using predicted motion field or temporal features to drive spatial features to deform and render frame by frame in the temporal dimension to generate a video sequence, preserving the original image details and handling complex environmental changes to ensure the temporal continuity of the video. By leveraging predicted motion field and temporal features, spatial features are driven to deform and render frame by frame in the temporal dimension to generate video sequences, ensuring smooth and natural transitions between frames; video details are preserved and enhanced, including retaining the fine texture of the original image during motion deformation, and maintaining high geometric consistency and realistic appearance of content from new perspectives when dynamically including viewpoint changes; complex environmental changes are handled, including background changes and scene-level events.

2. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The steps of static scene deep understanding and semantic deconstruction include: A deep neural network is used to extract multi-level deep features from the input static image. The deep neural network includes CNN or ViT. Identify and segment foreground objects, background environment, texture regions, lighting information, and spatial layout in an image, and annotate with semantic labels; Based on semantic tags combined with prior knowledge bases or physical models, we infer potential motion patterns or trends in a scene.

3. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: In the dynamic representation and modeling step, spatial features and temporal features in the depth features are separated. Spatial features are used to preserve the texture, color, and shape details of static elements, while temporal features are used to drive dynamic evolution.

4. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The temporal coherence generation in the high-fidelity video synthesis step is achieved by predicting motion field and temporal features, driving spatial features to deform and render frame by frame, ensuring smooth and natural transitions between frames, and avoiding flickering, jittering, or discontinuous motion.

5. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The detail preservation and enhancement in the high-fidelity video synthesis process includes: Texture preservation: During motion deformation, the fine textures in the original image are preserved and passed to subsequent frames; New perspective synthesis: When dynamics involve changes in perspective, the content under the new perspective maintains high geometric consistency and appearance realism.

6. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The handling of complex environmental changes in the high-fidelity video synthesis step includes: Background change processing: Processing cloud flow, water ripples, weather changes, and changes in light and shadow over time; Scene-level event handling: Simulates scene-level events such as flame combustion, smoke diffusion, particle effects, and plant growth.

7. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The high-fidelity video synthesis also includes a model architecture and training strategy. The model architecture adopts a generative model, which can perform multi-scale and spatiotemporal joint discrimination, judging both the authenticity of a single frame and the coherence between frames.

8. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The generative model includes a motion prediction module, which employs an optical flow prediction network, a deformation field prediction network, or a Transformer-based temporal modeling module.

9. The method for generating high-fidelity dynamic scene video based on a single static image according to claim 1, characterized in that: The generative model also includes a feature fusion and rendering module, which uses deformable convolution, optical flow-based feature warping, or neural radiation field techniques to transform and render spatial features according to temporal features.

Images