Video motion magnification method based on double-condition diffusion network

Abstract

The application discloses a video motion amplification method based on a double-condition diffusion network, relates to the technical field of computer vision, and obtains an original interframe motion field, constructs a target motion field in combination with a preset amplification multiple, generates target motion field prior embedding and starting frame appearance embedding; in a training stage, a diffusion mechanism is used to add noise to a real amplification frame to obtain a noisy real amplification frame, the spatial features of the noisy real amplification frame, the target motion field prior embedding and the starting frame appearance embedding are fused, and a time step embedding is combined to denoise the fused features to obtain a predicted amplification frame; in an inference generation stage, a noisy image is initialized as a reverse denoising starting point, the noisy image is gradually optimized through iterative denoising, posterior mean calculation and random noise sampling, and finally a motion amplification video frame is generated. Through the double driving of the motion field and the appearance features, the application solves the ringing artifact and excessive blur problems in the motion amplification video, so that high-precision video motion amplification is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and video processing technology, and in particular to a video motion amplification method based on a dual-conditional diffusion network. Background Technology

[0002] Video motion magnification technology aims to amplify minute movements in videos that are imperceptible to the naked eye to a perceptible level. As a cutting-edge computer vision technology, its core mission is to enhance the visualization of these minute spatial displacements or color changes. In practical applications, video motion magnification technology has demonstrated immense value: enhancing subtle movements in micro-expression recognition tasks, significantly improving the accuracy of micro-expression recognition; enabling high-precision non-contact medical monitoring in the medical field; and replacing traditional sensors to capture minute defects in industrial and structural inspection.

[0003] In recent years, the rapid development of computer vision and deep learning has driven continuous progress in video motion magnification technology. Deep learning-driven methods have shown significant advantages in model generalization and robustness, but they still inevitably face several key challenges: I. Existing video motion magnification methods mainly focus on representation learning. These methods typically decompose the video into shape and texture representations, and then use the difference between the shape representations to model and magnify the motion. However, this traditional paradigm of separation and reconstruction is often accompanied by insufficient decoupling of representations, which easily leads to the loss or degradation of texture information during the separation process, ultimately producing obvious ringing artifacts and excessive blurring in the reconstructed motion-magnified video.

[0004] For example, in 2024, Kwon Byung-Ki et al. published a paper titled "Learning-based Axial Video Motion Magnification" at the top international conference Proceedings of the European Conference on Computer Vision, which first proposed an axial video motion magnification method based on representation decoupling. This method decomposes the video into shape and texture representations, where motion information is modeled using the difference between shape representations, and the motion-magnified video is reconstructed by combining the difference between the magnified shape representations and the texture representation. Such methods often suffer from common problems: insufficient representation separation, meaning that interference between shape and texture representations can lead to the texture representation being incorrectly encoded into the shape representation; and texture representation degradation, meaning that information from the texture representation is lost during decomposition and reconstruction. These problems often result in excessive blurring in the reconstructed motion-magnified video.

[0005] Second, current optical flow-based video motion magnification methods often rely excessively on the absolute accuracy of optical flow data. Since the optical flow extracted by existing optical flow estimators inevitably contains noise, directly using these low-quality optical flows for network guidance or backpropagation will seriously interfere with model training. The lack of robust encoding of motion field features leads to motion leakage and ringing artifacts in the magnified video.

[0006] For example, in 2023, Zhaoying Pan et al. published a paper titled "Self-Supervised Motion Magnification by Backpropagating Through Optical Flow" at the top international conference Conference on Neural Information Processing Systems. This paper proposed a self-supervised motion magnification method based on optical flow backpropagation. This method uses a pre-trained optical flow network as part of the loss function for backpropagation, leveraging the consistency of optical flow to supervise the generative network, thus achieving motion magnification without requiring real magnified label data. However, this method directly uses low-quality optical flow extracted by the optical flow estimator as the core supervision signal. The inherent noise in the optical flow directly and unimpededly interferes with the model's training process, resulting in motion leakage. This demonstrates the limitations of traditional methods in handling optical flow, leading to noticeable ringing artifacts in the reconstructed motion-magnified video. Summary of the Invention

[0007] To overcome the shortcomings of the prior art, this invention provides a video motion magnification method based on a dual-conditional diffusion network. By dual-driven by motion field and appearance features, it solves the problems of ringing artifacts and excessive blurring in motion magnification videos, thereby achieving high-precision video motion magnification.

[0008] To achieve the above objectives, the present invention adopts the following technical solution, including: A video motion amplification method based on a dual-conditional diffusion network includes a training phase and an inference generation phase; The training phase includes: constructing a dual-condition driven diffusion network, which comprises a target motion field encoder, a starting frame appearance feature extractor, and a motion magnification frame generator; acquiring continuous images from the video, extracting the original inter-frame motion field between the starting frame and the current frame, and constructing the target motion field by combining it with a preset magnification factor; the target motion field encoder and the starting frame appearance feature extractor respectively encode the target motion field and the starting frame to obtain the target motion field prior embedding and the starting frame appearance embedding; the motion magnification frame generator adds noise to the real magnified frame of the current frame based on the diffusion mechanism to obtain a noisy real magnified frame, and fuses the spatial features of the noisy real magnified frame, the target motion field prior embedding, and the starting frame appearance embedding to obtain a fused feature, and combines the temporal step embedding to denoise the fused feature to obtain the predicted magnified frame of the current frame; and iteratively optimizing the network parameters based on the error between the predicted magnified frame and the real magnified frame to obtain the optimal network model. The inference generation stage includes: extracting the original motion field between frames based on the video frames to be processed, obtaining the target motion field by combining the target magnification factor, and generating the target motion field prior embedding and the starting frame appearance embedding accordingly; initializing the noisy image as the starting point for reverse denoising, and gradually optimizing the noisy image through iterative denoising, posterior mean calculation and random noise sampling, and finally generating the motion magnified video frame.

[0009] Preferably, the method for extracting the original motion field between frames is as follows: using an optical flow estimator to extract the starting frame. With the current frame Inter-frame original motion field The target motion field is constructed as follows: calculate the original motion field between frames. With magnification The product of these two factors gives the target sports field. ; where superscript This indicates the current training batch.

[0010] Preferably, the target motion field encoder and the starting frame appearance feature extractor respectively analyze the target motion field. , start frame Feature encoding is performed to obtain the prior embedding of the target motion field. and starting frame appearance embedding Specifically: ; ; in, The target motion field encoder uses a stacked residual convolutional network, which includes convolution operations with a kernel size of 3×3 and the GELU activation function to encode the deep spatial context information of the motion field and map it to the specified hidden layer dimension. This indicates the initial frame appearance feature extractor. The first layer uses a large receptive field convolution operation with a kernel size of 7×7, followed by a residual module for feature downsampling to capture global texture and structural information of the video frame; superscript This indicates the current training batch.

[0011] Preferably, the motion magnification frame generator adds noise to the real magnification frame to obtain a noisy real magnification frame, specifically as follows: Generate and real magnified frames Standard Gaussian noise with the same shape dimensions ; Define the maximum diffusion time step as From uniform distribution Random sampling diffusion time step ; For real magnified frames Adding Gaussian noise yields a true amplified frame of noise. : ; in, This indicates that the motion magnification frame generator is at the diffusion time step. Preset cumulative variance multiplication coefficient; superscript This indicates the current training batch.

[0012] Preferably, the motion magnification frame generator fuses the spatial features of the noisy real magnification frame, the prior embedding of the target motion field, and the appearance embedding of the starting frame to obtain fused features. The fused features are then combined with temporal step embedding to perform denoising processing to obtain the predicted magnification frame of the current frame. Specifically: ; in, As a feature of fusion, Amplified frames for noise Spatial characteristics, Represents a spatial feature extractor. Prior embedding for the target sports field. Embed the appearance of the starting frame. This indicates that alignment and stacking operations are performed along the channel dimensions. The superscript indicates a convolutional layer used for feature fusion and channel mapping. Indicates the current training batch; diffusion time step Encode to obtain time step embedding ;in, This represents a time-step feature encoder; ; in, To predict the magnified frame, This indicates a diffusion denoiser.

[0013] Preferably, the predicted magnified frame With the corresponding real magnified frame Perform feature and pixel-level comparison, and calculate the first step of the diffusion network. Training error during training : ; in, The superscript represents the mean absolute error loss function. This indicates the current training batch.

[0014] Preferably, the specific method for the reasoning generation stage is as follows: Get the starting frame of the video to be processed Current frame and magnification Extract the start frame With the current frame Inter-frame original motion field And generate the corresponding prior embedding of the target motion field. Embedded with the appearance of the start frame ; where superscript Indicates the reasoning generation stage; Initialize the current diffusion time step Sample the current diffusion time step from the standard normal distribution, which has the same dimension as the target video frame. Noisy images , as the starting point for reverse denoising; embed the prior of the target sports field , Start Frame Appearance Embedding and noisy images By fusing spatial features, the current diffusion time step is obtained. Fusion characteristics ; Diffusion time step Encode to obtain time step embedding Combined with time step embedding For the current diffusion time step Fusion characteristics Denoising is performed to obtain the current diffusion time step. Corresponding prediction magnified frame ; via the current diffusion time step Corresponding prediction magnified frame and noisy images Calculate diffusion time step posterior distribution mean : ; in, For the current diffusion time step variance , For the current diffusion time step The cumulative multiplication factor; Random noise is introduced for sampling to obtain the diffusion time step. Noisy images : ; in, For the current diffusion time step standard deviation, take ; For random noise, when At that time, no random noise is added, that is ; like The corresponding predicted magnified frame is then obtained. For the final generated motion-magnified video frames; if Then let , Continue noise reduction processing until... .

[0015] The present invention also provides a readable storage medium having a computer program stored thereon, which, when executed, implements the aforementioned video motion amplification method based on a dual-conditional diffusion network.

[0016] The present invention also provides an electronic device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned video motion amplification method based on a dual-conditional diffusion network.

[0017] The present invention also provides a computer program product comprising a computer program / instruction that, when executed by a processor, implements the aforementioned video motion amplification method based on a dual-conditional diffusion network.

[0018] The advantages of this invention are: (1) This invention introduces a video motion magnification mechanism based on dual driving of motion field and appearance features. By using a motion magnification frame generator, it effectively solves the problems of ringing artifacts and blurring in motion magnification videos caused by texture degradation in traditional methods, thereby achieving high-precision video motion magnification.

[0019] (2) This invention innovatively proposes a dual-condition guidance mechanism to control the generation process of the diffusion network. The target motion field prior (guiding motion deformation) and the appearance of the starting frame (preserving texture structure) are clearly defined as dual control conditions. By embedding the target motion field prior, embedding the starting frame appearance, and stacking and convolutionally fusing the spatial features of the noisy image in the channel dimension, these dual conditions can deeply intervene in each time step of the diffusion denoiser. Through feature-level intervention, the diffusion model is precisely controlled to generate video frames that take into account both accurate motion shape and high-fidelity texture, providing rigorous physical and visual dual constraints for the subsequent reverse generation process.

[0020] (3) This invention introduces a method based on The iterative denoising generation strategy of the prediction paradigm, by combining the reparameterized derivation of DDPM to calculate the posterior distribution mean and introducing random noise for rigorous iterative sampling, can stably and with high quality generate high-precision motion magnification video frames that ultimately eliminate ringing artifacts. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the training process of the present invention.

[0022] Figure 2 This is a schematic diagram of the reasoning process of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The present invention provides a video motion magnification method based on a dual-conditional diffusion network. By driving the motion field and appearance features, it effectively solves the problems of ringing artifacts and excessive blurring in the reconstructed motion magnification video, thereby generating high-precision motion magnification video.

[0025] The present invention specifically includes the following steps.

[0026] Step 1: Construct a dual-condition driven diffusion network, which includes: a target motion field encoder, a starting frame appearance feature extractor, a motion magnification frame generator, and a target motion field encoder.

[0027] like Figure 1 As shown, the training phase is performed, and the current training batch is defined as... and initialize , will the The network parameters during training are denoted as follows: In this embodiment, the AdamW optimizer is used for training, the initial learning rate is set to 0.0001, and a cosine annealing strategy is used to decay the learning rate.

[0028] Step 2: Acquire consecutive images from the video, and denote the first frame as the [missing frame number]. The starting frame during the second training session ; denote the subsequent frame as the first The current frame during the next training session For subsequent frames (current frame) Perform motion magnification; zoom in on the current frame. The corresponding real magnified frame is denoted as ; Enlarge the real frame The magnification factor is denoted as The starting frame is extracted using an optical flow estimator (a pre-trained RAFT model to extract motion fields with high-precision edges). With the current frame Inter-frame original motion field .

[0029] Step 3: Calculate the original motion field between frames using equation (1). With magnification The product of these factors is used to construct the target sports field. : (1) Step 4: The target motion field encoder uses formula (2) to scan the target motion field. Perform feature encoding to generate the prior embedding of the target motion field. : (2) In equation (2), This represents the target motion field encoder, which employs a stacked residual convolutional network containing convolution operations with a kernel size of 3×3 and the GELU activation function to encode the deep spatial context information of the motion field and map it to a specified hidden dimension.

[0030] Step 5: The starting frame appearance feature extractor uses equation (3) to extract the appearance feature of the video starting frame. Perform feature extraction to generate the initial frame appearance embedding. : (3) In equation (3), This represents the appearance feature extractor for the initial frame. To better capture global texture and structural information, the first layer of this extractor uses a large receptive field convolution operation with a kernel size of 7×7, followed by a residual module for feature downsampling.

[0031] Step 6: The motion magnification frame generator uses equation (4) to obtain the first... Training time and real magnified frames Standard Gaussian noise with the same shape dimensions This provides a clean noise distribution for the subsequent forward diffusion process. (4) In equation (4), This indicates the operation of generating Gaussian noise.

[0032] Step 7: Randomly sample diffusion time steps from a uniform distribution. ,in For the maximum diffusion time step; the motion magnification frame generator uses equation (5) to generate the real magnification frame. Adding Gaussian noise yields the first The noise was realistically amplified during the second training session. : (5) In equation (5), This indicates that the motion magnification frame generator is at the diffusion time step. The preset cumulative variance multiplication coefficient is used, and in this embodiment, a linear schedule strategy is adopted to control the proportion of noise injection.

[0033] Step 8: The motion magnification frame generator uses equation (6) to embed the prior of the target motion field. , Start Frame Appearance Embedding and noise-enhanced real frames The spatial features are aligned and stacked along the channel dimension, and then fused using a convolutional network to obtain the first... Fusion features during training : (6) In equation (6), Indicates the first Spatial feature extractor during training. Indicates the first The noise was realistically amplified during the second training session. Spatial characteristics, This indicates that alignment and stacking operations are performed along the channel dimensions. Indicates the first The convolutional layers used for feature fusion and channel mapping during training. Here... Pointwise convolution with a kernel size of 1×1 is used to effectively aggregate pixel-level cross-channel local features and eliminate redundancy in channel dimensions, thereby achieving feature-level control and guidance of the diffusion generation process by dual prior conditions.

[0034] Step 9: The motion magnification frame generator uses equation (7) to adjust the diffusion time step. Encode to obtain time step embedding The motion magnification frame generator utilizes equation (8) to fuse features. Perform denoising operation to predict the first... Magnified frames during the second training session : (7) (8) In equation (7), Indicates the first The time-step feature encoder during training uses sinusoidal positional encoding combined with a multilayer perceptron to generate high-dimensional vectors; in equation (8), Indicates the first For the diffusion denoising during training, this embodiment uses the U-Net network skeleton with self-attention mechanism to extract features.

[0035] Step 10: Using the prediction paradigm from the diffusion model domain (i.e., directly predicting the noise-free target image), the first... Magnified frames predicted by the diffusion network during training. With the corresponding real magnified frame Perform feature and pixel-level comparison, and use Equation (9) to calculate the first step of the diffusion network. Overall training error during training : (9) In equation (9), This represents the mean absolute error loss function; in video motion magnification tasks, minute movements are often accompanied by high-frequency details, and the mean absolute error loss... Compared to mean square error loss It can effectively alleviate the problem of blurry generated images and better protect image edges.

[0036] Step 11, determine the first Error during training Does it converge? If it converges, then determine the first... Network parameters during training For optimal parameters and the optimal parameters After the corresponding diffusion network is used as the final model for generating motion-amplified video frames, step 13 is executed; if it does not converge, the diffusion network continues to execute step 13. Subgradient backpropagation to update network parameters and in obtaining the first Network parameters during training After determining the network parameters (specifically, the network parameters of the motion magnification frame generator), proceed to step 12.

[0037] Step 12, Assign to Then return to step 2 and execute sequentially.

[0038] like Figure 2 As shown, the process enters the reasoning generation stage.

[0039] Step 13: The parameters of the dual-condition driven diffusion network are optimal. ; Obtain the starting frame of the video to be processed Current frame and magnification The starting frame is extracted using an optical flow estimator. With the current frame Inter-frame original motion field Extract the prior embedding of the target motion field in the same manner as steps 3 to 5. Embedded with the appearance of the start frame ; superscript This indicates the testing phase, which is the reasoning generation phase.

[0040] Step 14: Initialize the current diffusion time step Noisy images of the current step with the same dimension as the target video frame are sampled from a standard normal distribution. This serves as the starting point for reverse denoising in the diffusion model: ; Step 15: Perform iterative denoising, and embed the prior of the target motion field using equation (10). , Start Frame Appearance Embedding and the noisy image of the current step Spatial features are aligned and stacked along the channel dimension, and then fused using a trained convolutional network to obtain the fused features required for inference. This fusion feature carries dual conditional information on motion and appearance, used to control the denoising direction; the diffusion time step is used in equation (11). Encode to obtain time step embedding ; Use equation (12) to fuse the features of the current diffusion time step and The input is fed into the trained diffusion denoiser to predict the magnified frame corresponding to the current step. : (10) (11) (12) in, This represents the trained spatial feature extractor. This represents the time-step feature encoder. This represents a trained convolutional layer. This represents a trained diffusion denoiser.

[0041] Step 16: Calculate the posterior mean. Based on the reparameterization derivation of DDPM, use equation (13) to predict the mean. and the current Calculate time steps posterior distribution mean : (13) In equation (13), For time step variance , This is the cumulative multiplication factor.

[0042] Step 17: Introduce random noise for sampling using equation (14) (when...) At that time, no random noise is added, that is ): (14) In equation (14), , For time step standard deviation, take ; like Then the result is This refers to the final generated high-precision motion-magnified video frame that has had ringing artifacts eliminated; if Then let , Return to step 15 and execute sequentially.

[0043] To objectively verify the method of this invention, this embodiment uses the Structural Similarity Index Measure (SSIM) as the evaluation index for quantitative experiments, and performs quantitative analysis on motion-amplified videos on a motion-amplified synthetic dataset. The formula for calculating SSIM is as follows: ; in, This represents the motion-magnified video frames obtained through inference. , express The corresponding real-world magnified video frame; They represent and The average pixel value; They represent and The pixel variance; express and covariance; It is a constant added to maintain the stability of the formula calculation (to prevent the denominator from approaching 0); SSIM generally takes the value range of [0,1] in the image domain, and the closer it is to 1, the closer the motion magnified video frame is to the real magnified video frame.

[0044] On the motion magnification synthesis dataset, the SSIM values ​​of different methods at different magnification factors were calculated, and the results are shown in Table 1 below. The SSIM values ​​of the method of the present invention are higher than those of other methods at each magnification factor, indicating that the motion magnification video frames generated by the method of the present invention can accurately restore the structural details of real magnified video frames, which fully demonstrates the practical effect of the method of the present invention in eliminating edge artifacts and generating high-quality videos.

[0045] Table 1 SSIM values ​​at different magnifications

[0046] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A video motion amplification method based on a dual-conditional diffusion network, characterized in that, Includes the training phase and the inference generation phase; The training phase includes: constructing a dual-condition driven diffusion network, which comprises a target motion field encoder, a starting frame appearance feature extractor, and a motion magnification frame generator; acquiring continuous images from the video, extracting the original inter-frame motion field between the starting frame and the current frame, and constructing the target motion field by combining it with a preset magnification factor; the target motion field encoder and the starting frame appearance feature extractor respectively encode the target motion field and the starting frame to obtain the target motion field prior embedding and the starting frame appearance embedding; the motion magnification frame generator adds noise to the real magnified frame of the current frame based on the diffusion mechanism to obtain a noisy real magnified frame, and fuses the spatial features of the noisy real magnified frame, the target motion field prior embedding, and the starting frame appearance embedding to obtain a fused feature, and combines the temporal step embedding to denoise the fused feature to obtain the predicted magnified frame of the current frame; and iteratively optimizing the network parameters based on the error between the predicted magnified frame and the real magnified frame to obtain the optimal network model. The inference generation stage includes: extracting the original motion field between frames based on the video frames to be processed, obtaining the target motion field by combining the target magnification factor, and generating the target motion field prior embedding and the starting frame appearance embedding accordingly; initializing the noisy image as the starting point for reverse denoising, and gradually optimizing the noisy image through iterative denoising, posterior mean calculation and random noise sampling, and finally generating the motion magnified video frame.

2. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, The method for extracting the original motion field between frames is as follows: using an optical flow estimator to extract the starting frame. With the current frame Inter-frame original motion field The target motion field is constructed as follows: calculate the original motion field between frames. With magnification The product of these two factors gives the target sports field. ; where superscript This indicates the current training batch.

3. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, The target motion field encoder and the starting frame appearance feature extractor respectively analyze the target motion field. , start frame Feature encoding is performed to obtain the prior embedding of the target motion field. and starting frame appearance embedding Specifically: in, The target motion field encoder uses a stacked residual convolutional network, which includes convolution operations with a kernel size of 3×3 and the GELU activation function to encode the deep spatial context information of the motion field and map it to the specified hidden layer dimension. This indicates the initial frame appearance feature extractor. The first layer uses a large receptive field convolution operation with a kernel size of 7×7, followed by a residual module for feature downsampling to capture global texture and structural information of the video frame; superscript This indicates the current training batch.

4. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, The motion magnification frame generator adds noise to the real magnification frame to obtain a noisy real magnification frame, specifically: Generate and real magnified frames Standard Gaussian noise with the same shape dimensions ; Define the maximum diffusion time step as From uniform distribution Random sampling diffusion time step ; For real magnified frames Adding Gaussian noise yields a true amplified frame of noise. : in, This indicates that the motion magnification frame generator is at the diffusion time step. Preset cumulative variance multiplication coefficient; superscript This indicates the current training batch.

5. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, The motion magnification frame generator fuses spatial features of the noisy real magnification frame, prior embedding of the target motion field, and appearance embedding of the starting frame to obtain fused features. These fused features are then combined with temporal step embedding to perform denoising processing, resulting in the predicted magnification frame for the current frame. Specifically: in, As a feature of fusion, Amplified frames for noise Spatial characteristics, Represents a spatial feature extractor. Prior embedding for the target sports field. Embed the appearance of the starting frame. This indicates that alignment and stacking operations are performed along the channel dimensions. The superscript indicates a convolutional layer used for feature fusion and channel mapping. Indicates the current training batch; diffusion time step Encode to obtain time step embedding ;in, This represents a time-step feature encoder; in, To predict the magnified frame, This indicates a diffusion denoiser.

6. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, Predicted upscaled frames With the corresponding real magnified frame Perform feature and pixel-level comparison, and calculate the first step of the diffusion network. Training error during training : in, The superscript represents the mean absolute error loss function. This indicates the current training batch.

7. The video motion amplification method based on a dual-conditional diffusion network according to claim 1, characterized in that, The specific method of the reasoning generation stage is as follows: Get the starting frame of the video to be processed Current frame and magnification Extract the start frame With the current frame Inter-frame original motion field And generate the corresponding prior embedding of the target motion field. Embedded with the appearance of the start frame ; where superscript Indicates the reasoning generation stage; Initialize the current diffusion time step Sample the current diffusion time step from the standard normal distribution, which has the same dimension as the target video frame. Noisy images , as the starting point for reverse denoising; embed the prior of the target sports field , Start Frame Appearance Embedding and noisy images By fusing spatial features, the current diffusion time step is obtained. Fusion characteristics ; Diffusion time step Encode to obtain time step embedding Combined with time step embedding For the current diffusion time step Fusion characteristics Denoising is performed to obtain the current diffusion time step. Corresponding prediction magnified frame ; via the current diffusion time step Corresponding prediction magnified frame and noisy images Calculate diffusion time step posterior distribution mean : in, For the current diffusion time step variance , For the current diffusion time step The cumulative multiplication factor; Introducing random noise for sampling Diffusion time step Noisy images : in, For the current diffusion time step standard deviation, take ; For random noise, when At that time, no random noise is added, that is ; like The corresponding predicted magnified frame is then obtained. For the final generated motion-magnified video frames; if Then let , Continue noise reduction processing until... .

8. A readable storage medium, characterized in that, It stores a computer program, which, when executed, implements a video motion amplification method based on a dual-conditional diffusion network as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a video motion amplification method based on a dual-conditional diffusion network as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the video motion amplification method based on a dual-conditional diffusion network as described in any one of claims 1 to 7.

Images