Video motion amplification method based on cyclic generative adversarial network

Abstract

The invention relates to the technical field of computer vision and video processing, in particular to a video motion amplification method based on a cyclic generative adversarial network, which comprises the following steps: inputting a reference frame, a query frame and a motion amplification factor into a trained video motion amplification model for motion amplification; during training, an amplification frame is constructed and generated through the motion amplification module; constructing and generating a reference frame through a motion reduction module; discriminating the difference between the generated amplified frame and the corresponding real amplified frame through a first discriminator; discriminating the difference between the generated reference frame and the corresponding reference frame through a second discriminator; and a fusion loss calculation module calculates fusion loss and optimizes network parameters by combining the difference between the generated amplified frame and the corresponding real amplified frame and the difference between the generated reference frame and the corresponding reference frame according to the discrimination results of the first discriminator and the second discriminator. According to the method, high fidelity and high robustness can be realized in various complex scenes, and video motion amplification of artifacts can be effectively inhibited.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and video processing technology, and specifically to a video motion amplification method based on a recurrent generative adversarial network. Background Technology

[0002] Vibration is a ubiquitous phenomenon in nature, permeating every aspect of our daily lives, from the microscopic to the macroscopic, from the subtle to the significant. It is not only a direct manifestation of the movement of matter but also a crucial tool for conveying information and unlocking the mysteries of the world. The measurement of vibration signals is of great importance in many fields because it can provide valuable insights into the internal dynamics of structural, mechanical, and biological systems. Traditional measurement methods, which typically rely on contact sensors such as accelerometers or strain gauges, have certain limitations.

[0003] Visual measurement technology offers a flexible solution for measuring vibration signals, capturing vibration information across the entire field of view without disturbing the object being measured. However, visual measurement still faces numerous challenges when dealing with extremely small movements. Tiny vibration signals are often masked by visual noise or fall below the sensitivity threshold of the camera equipment, leading to missed detections or insufficient measurement accuracy. To address these issues, amplifying these minute movements before data processing is crucial. This not only ensures their clear visibility but also significantly improves measurement accuracy. By amplifying these minute movements, we aim to enhance the sensitivity of the visual measurement system for more accurate and reliable analysis of subtle vibration signals.

[0004] Video motion magnification technology aims to reveal and enhance minute movements in video sequences that are imperceptible to the naked eye. It functions like a "visual microscope," with crucial applications in structural health monitoring, biomedical analysis, micro-expression recognition, and industrial non-destructive testing. The core objective of this technology is to extract weak motion signals from video data and then visualize and magnify them through algorithmic strategies, thereby revealing dynamic patterns hidden within the scene. The rise of deep learning has brought a new paradigm to video motion magnification. Learning-based methods aim to learn the complex mapping functions required for motion magnification directly from the data using architectures such as convolutional neural networks or Transformers, thereby reducing reliance on manually designed filters.

[0005] The applicant's research revealed that while deep learning-based methods have achieved significant performance improvements over traditional methods, they still face several serious challenges. First, these models typically require large amounts of high-quality labeled data for training, and constructing precisely paired motion-magnified video datasets is both costly and extremely difficult. Second, many methods employ a unidirectional forward inference architecture, directly generating magnified frames from input frames, lacking effective constraints on the physical plausibility of the motion magnification process. This easily leads to spatiotemporally inconsistent artifacts, such as inter-frame flickering, motion distortion, and unrealistic texture deformation. Third, training deep networks for fine motion manipulation is highly susceptible to mode collapse, where the generator tends to ignore subtle movements and output static or repetitive content, resulting in magnification failure. Finally, existing methods have limited generalization capabilities; their performance significantly degrades when faced with new scenes and motion patterns outside the training data distribution.

[0006] In summary, existing technologies, whether relying on traditional signal processing methods with manually manipulated parameters or deep learning methods driven by large-scale data, all have inherent limitations in terms of generalization ability, artifact resistance, training stability, and dependence on paired data. Therefore, there is an urgent need in this field for a novel technical solution that can ensure physical plausibility, high fidelity, and strong generalization ability in motion amplification without relying on precise paired data. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, the technical problem to be solved by this invention is: how to provide a video motion amplification method based on a cyclic generative adversarial network, by introducing a bidirectional closed-loop motion control framework to organically combine the motion amplification and motion reduction processes, and using cyclic consistency constraints to ensure the physical rationality and reversibility of motion transformation, thereby achieving high-fidelity, high-robustness video motion amplification with effective artifact suppression in various complex scenarios.

[0008] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0009] A video motion amplification method based on recurrent generative adversarial networks includes:

[0010] S1: Obtain the video sequence data to be processed, and determine the reference frame and query frame in the video sequence data;

[0011] S2: Input the reference frame, query frame and motion magnification factor into the trained video motion magnification model for motion magnification processing, and output the generated magnified frame after motion magnification.

[0012] The processing steps for training a video motion magnification model include:

[0013] S201: Obtain the reference frame, query frame, motion magnification factor and motion reduction factor used as training data, as well as the real magnified frame corresponding to the query frame as the training input for the video motion magnification model;

[0014] S202: The motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame after motion magnification;

[0015] S203: The motion reduction module reduces the feature differences between the generated magnified frame and the query frame based on the motion reduction factor, and fuses them with the features of the query frame to construct a generated reference frame corresponding to the reference frame.

[0016] S204: The first discriminator determines the difference between the generated magnified frame and the corresponding real magnified frame;

[0017] S205: The second discriminator determines the difference between the generated reference frame and the corresponding reference frame;

[0018] S206: The fusion loss calculation module calculates the fusion loss based on the discrimination results of the first discriminator and the second discriminator, combined with the difference between the generated magnified frame and the corresponding real magnified frame and the difference between the generated reference frame and the corresponding reference frame, and optimizes the network parameters of the video motion magnification model based on the fusion loss.

[0019] S207: Repeat steps S201 to S206 to iteratively train the video motion magnification model until the model converges or reaches the maximum number of iterations.

[0020] After the video motion magnification model is trained, the motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame as the output of the video motion magnification model.

[0021] Preferably, in step S202, the motion amplification module is a generator network comprising a cascaded encoder, manipulator, and decoder;

[0022] The encoder is used to extract multi-scale features from the reference frame and the query frame respectively, and obtain shallow features, medium features and deep features of the reference frame and the query frame.

[0023] The manipulator is used to calculate the feature differences between the deep features of the reference frame and the query frame, and to weight these feature differences through a channel attention mechanism to obtain channel-enhanced features. After amplifying the channel-enhanced features through a motion magnification factor, they are fused with the deep features of the query frame to obtain the manipulated features.

[0024] The decoder is used to fuse the manipulated features with the shallow and mid-level feature maps of the query frame step by step to construct the generated magnified frame after motion magnification.

[0025] Preferably, in step S202, the encoder includes cascaded padding layers and three convolutional layers; the convolutional layers include cascaded convolutional modules and ReLU activation functions;

[0026] The three convolutional layers are used to output shallow, medium, and deep features of the reference frame, respectively.

[0027] Preferably, in step S202, the calculation formula for the manipulator is expressed as follows:

[0028] M(X a ,X b ,α)=ftB2+α*Attention(ftB2-ftA2);

[0029] In the formula: M(·) represents the manipulator; Attention(·) represents the channel attention operation, used to emphasize the feature channels most relevant to motion transformation; ftA2 represents the reference frame X. a The deep features; ftB2 represents the query frame X b The deep features; α represents the motion amplification factor.

[0030] Preferably, in step S202, the decoder includes several cascaded enhanced residual blocks, a first upsampling module, a first convolutional layer, a second upsampling module, a second convolutional layer, a first padding layer, and a third convolutional layer; the output of the second convolutional layer is fused with the mid-level features of the query frame and then input to the first padding layer; the output of the third convolutional layer is fused with the shallow features of the query frame and then used as the generated magnified frame after motion amplification of the decoder output.

[0031] The enhanced residual block consists of a cascaded second padding layer, a fourth convolutional layer, a third padding layer, a fifth convolutional layer, and a ReLU activation layer; the output of the fourth convolutional layer is either Dropout processed or identity processed before being input to the second padding layer; the output of the fifth convolutional layer is added element-wise to the input of the enhanced residual block before being input to the ReLU activation layer.

[0032] The first, second, and fourth convolutional layers consist of cascaded convolutional modules and ReLU activation functions; the third convolutional layer consists of cascaded convolutional modules and Tanh activation functions; and the fifth convolutional layer consists of convolutional modules.

[0033] Preferably, in step S204, the first discriminator is generated based on an improved Patch GAN:

[0034] Two cascaded dilated convolutional modules are connected after the last basic convolutional module of the Patch GAN backbone network, and the output of the second dilated convolutional module is used as the output of the backbone network.

[0035] The backbone network's output is fed into two paths: the first path is the original local discrimination path of Patch GAN, which outputs an N×N matrix, where each element corresponds to the authenticity judgment of a local region of the input video frame; the second path applies global average pooling to the features output by the backbone network to compress the feature map into a feature vector, and then outputs a single global authenticity scalar based on the feature vector through a fully connected layer.

[0036] Preferably, in step S206, the fusion loss calculation module calculates the fusion loss using the following formula:

[0037] L total =λ adv *L adv +λ cyc *L cyc +λ id *L id +λ pct *L pct ;

[0038] In the formula: L total L represents the fusion loss; adv L represents the sum of the discrimination loss of the first discriminator for generating the magnified frame and the discrimination loss of the second discriminator for generating the reference frame; cyc L represents the cycle consistency loss calculated based on the difference between the generated reference frame and the corresponding reference frame; id This represents the identity loss between the generated magnified frame and the actual magnified frame; L pct λ represents the perceptual loss between the generated magnified frame and the actual magnified frame; adv , λ cyc , λ id , λ pct This represents the weighting coefficient for each loss.

[0039] Preferably, the resistance to loss L adv The calculation formula is:

[0040]

[0041] Where: G mag (·) indicates the motion amplification module; X a Indicates a reference frame; X b This represents the query frame; α represents the motion magnification factor; Y represents the generated magnified frame; L represents the actual magnified frame. global Indicates the global discrimination path output of the discriminator; Lpatch This indicates the local path output of the discriminator.

[0042] Preferably, the cycle consistency loss L cyc The calculation formula is:

[0043]

[0044] In the formula: L1 represents the L1 norm; X a Indicates a reference frame; Indicates the generation of a reference frame; G mag (·) indicates the motion amplification module; G min (·) indicates the motion reduction module; X b Y represents the query frame; Y represents the actual magnified frame. This indicates the generation of a magnified frame; α represents the motion magnification factor.

[0045] Preferably, identity loss L id The calculation formula is:

[0046]

[0047] In the formula: L1 represents the L1 norm; G min (·) indicates the motion reduction module; X b Indicates a query frame; This indicates the generation of a magnified frame;

[0048] Perceived loss L pct The calculation formula is:

[0049]

[0050] In the formula: L2 represents the L2 norm; Y represents the generated magnified frame; L represents the actual magnified frame. LPIPS Learn to perceive image patch similarity loss.

[0051] Compared with existing technologies, the video motion amplification method based on recurrent generative adversarial networks in this invention has the following advantages:

[0052] Traditional methods generally face technical bottlenecks such as insufficient generalization ability and significant spatiotemporal artifacts when handling complex scenes and high magnification, while deep learning-based methods are constrained by the scarcity of paired data and unstable training. To address these core issues, this invention achieves a significant breakthrough in both technical principles and engineering practice by constructing a bidirectional closed-loop motion control framework. This scheme organically combines motion magnification and motion reduction processes, ensuring the physical rationality of motion transformations through cyclic consistency constraints, thus significantly improving the robustness and applicability of the method while maintaining high fidelity.

[0053] At the architectural design level, this invention introduces recurrent adversarial generative networks (RBANs) into the field of video motion magnification for the first time, creatively proposing the concept of "motion downscaling." This architectural innovation breaks through the limitations of traditional unidirectional generative models. Through strict cyclic consistency constraints, it enables the motion transformation process to possess physical reversibility, intrinsically ensuring the physical reversibility of motion transformation and significantly suppressing spatiotemporal artifacts (such as flickering and distortion), providing dual guarantees for the accuracy of the magnification factor and its visual effect. Particularly noteworthy is that this framework achieves self-supervised learning without the need for paired training data, which has significant value in practical applications, greatly expanding the application scope and data adaptability of the method.

[0054] In terms of technological innovation, the generator of this invention integrates U-Net skip connections into the encoder-manipulator-decoder framework. By bridging multi-scale encoder-decoder features, it simulates phase-based pyramid functionality, suppressing artifacts through a learnable hierarchical structure while preserving spatial fidelity, outperforming hand-designed methods. First, the channel attention module introduced in the generator adaptively focuses on key motion-related feature channels, achieving precise extraction and amplification of subtle movements. Second, the decoder, employing a U-Net structure, fuses deep and shallow features through skip connections, maintaining excellent spatial detail and edge fidelity while amplifying motion. Furthermore, the enhanced discriminator design, combining dilated convolution and a dual-output structure, balances local texture realism and global spatiotemporal consistency, effectively improving the stability and discriminative sensitivity of adversarial training. To address the challenge of training deep networks for subtle motion manipulation, this invention redesigns the Patch GAN discriminator. By incorporating dilated convolution and a hybrid discriminator combining global and local adversarial losses, the network becomes more sensitive to micro-motions, stabilizing adversarial training and effectively preventing mode collapse.

[0055] In terms of performance, this invention demonstrates significant advantages across multiple evaluation metrics. Extensive experimental verification showed that at 20x magnification, the method achieved an average MUSIQ score of 46.44, ranking first in 8 out of 10 test sequences. Even under extreme magnification conditions of 40-80x, this method maintains stable visual quality with no significant artifacts, exhibiting excellent robustness to complex scenes and large-scale motion.

[0056] In terms of application value, this invention provides an effective tool for solving technical problems in multiple professional fields. In the field of industrial inspection, the method of this invention can amplify minute mechanical vibrations and deformations, providing intuitive evidence for equipment condition monitoring and fault diagnosis; in medical diagnosis, it can clearly present physiological micro-movements such as vascular pulsation and tissue tremors, assisting doctors in making accurate diagnoses; in scientific research, it provides strong technical support for the visualization and analysis of minute dynamic processes in scientific research activities such as physical experiments and biological observations; in the field of security monitoring, it can enhance subtle activities in monitoring videos, improving the accuracy and timeliness of security warnings.

[0057] Finally, it is particularly important to note that the closed-loop motion control framework proposed in this invention has broad technical versatility. This method is not limited to traditional RGB video motion amplification, but can be extended to various imaging modalities such as infrared, microscopy, and remote sensing, providing a unified technical solution for small motion analysis in multiple fields. This versatility gives this invention a wider application prospect and a longer technological lifespan, laying a solid foundation for subsequent technological evolution and application innovation. Attached Figure Description

[0058] To make the objectives, technical solutions, and advantages of the invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0059] Figure 1 This is a network structure diagram of a video motion amplification method based on a recurrent generative adversarial network.

[0060] Figure 2 This is the network structure diagram for training the framework and discriminator.

[0061] Figure 3 This is an example diagram of the dataset provided in this embodiment.

[0062] Figure 4 Line graphs showing the MUSIQ values ​​of the eight methods provided in this embodiment across ten video sequences.

[0063] Figure 5 Line graph showing the average MUSIQ values ​​of the eight methods provided in this embodiment across 10 video sequences.

[0064] Figure 6 Line graphs showing the MUSIQ values ​​of the eight methods provided in this embodiment at different magnifications.

[0065] Figure 7 Line graph showing the overall performance of the MUSIQ values ​​for the eight methods provided in this embodiment.

[0066] Figure 8 This is a comparison chart to verify the effectiveness of the training framework provided in this embodiment.

[0067] Figure 9 This is a comparison chart showing the application of the eight methods provided in this embodiment to the fixed-line video sequence "balloon".

[0068] Figure 10 This is a comparison chart of the eight methods provided in this embodiment on the "bird" video sequence. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but only to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0070] The following detailed explanation illustrates the specific implementation methods:

[0071] Example:

[0072] Traditional video motion magnification methods suffer from significant artifacts, weak generalization ability, and reliance on paired data when handling large movements, complex scenes, or high magnification. While existing deep learning-based methods offer some improvement, their unidirectional generative architecture lacks effective constraints on the physical plausibility of the motion process and suffers from unstable training. To address these issues and shortcomings, this invention proposes a video motion magnification method based on recurrent adversarial generative networks (ROVs). The core of this method lies in constructing a bidirectional closed-loop training framework that couples the motion magnification and reduction processes through a recurrent consistency loss, thereby achieving high-fidelity and controllable magnification of subtle movements.

[0073] like Figure 1 As shown, a video motion amplification method based on recurrent generative adversarial networks includes:

[0074] S1: Obtain the video sequence data to be processed, and determine the reference frame and query frame in the video sequence data;

[0075] S2: Input the reference frame, query frame and motion magnification factor into the trained video motion magnification model for motion magnification processing, and output the generated magnified frame after motion magnification.

[0076] The processing steps for training a video motion magnification model include:

[0077] S201: Obtain the reference frame, query frame, motion magnification factor and motion reduction factor used as training data, as well as the real magnified frame corresponding to the query frame as the training input for the video motion magnification model;

[0078] S202: The motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame after motion magnification;

[0079] S203: The motion reduction module reduces the feature differences between the generated magnified frame and the query frame based on the motion reduction factor, and fuses them with the features of the query frame to construct a generated reference frame corresponding to the reference frame.

[0080] S204: The first discriminator determines the difference between the generated magnified frame and the corresponding real magnified frame;

[0081] S205: The second discriminator determines the difference between the generated reference frame and the corresponding reference frame;

[0082] S206: The fusion loss calculation module calculates the fusion loss based on the discrimination results of the first discriminator and the second discriminator, combined with the difference between the generated magnified frame and the corresponding real magnified frame and the difference between the generated reference frame and the corresponding reference frame, and optimizes the network parameters of the video motion magnification model based on the fusion loss.

[0083] S207: Repeat steps S201 to S206 to iteratively train the video motion magnification model until the model converges or reaches the maximum number of iterations.

[0084] After the video motion magnification model is trained, the motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame as the output of the video motion magnification model.

[0085] Traditional methods generally face technical bottlenecks such as insufficient generalization ability and significant spatiotemporal artifacts when handling complex scenes and high magnification, while deep learning-based methods are constrained by the scarcity of paired data and unstable training. To address these core issues, this invention achieves a significant breakthrough in both technical principles and engineering practice by constructing a bidirectional closed-loop motion control framework. This scheme organically combines motion magnification and motion reduction processes, ensuring the physical rationality of motion transformations through cyclic consistency constraints, thus significantly improving the robustness and applicability of the method while maintaining high fidelity.

[0086] At the architectural design level, this invention introduces recurrent adversarial generative networks (RBANs) into the field of video motion magnification for the first time, creatively proposing the concept of "motion downscaling." This architectural innovation breaks through the limitations of traditional unidirectional generative models. Through strict cyclic consistency constraints, it enables the motion transformation process to possess physical reversibility, intrinsically ensuring the physical reversibility of motion transformation and significantly suppressing spatiotemporal artifacts (such as flickering and distortion), providing dual guarantees for the accuracy of the magnification factor and its visual effect. Particularly noteworthy is that this framework achieves self-supervised learning without the need for paired training data, which has significant value in practical applications, greatly expanding the application scope and data adaptability of the method.

[0087] In terms of technological innovation, the generator of this invention integrates U-Net skip connections into the encoder-manipulator-decoder framework. By bridging multi-scale encoder-decoder features, it simulates phase-based pyramid functionality, suppressing artifacts through a learnable hierarchical structure while preserving spatial fidelity, outperforming hand-designed methods. First, the channel attention module introduced in the generator adaptively focuses on key motion-related feature channels, achieving precise extraction and amplification of subtle movements. Second, the decoder, employing a U-Net structure, fuses deep and shallow features through skip connections, maintaining excellent spatial detail and edge fidelity while amplifying motion. Furthermore, the enhanced discriminator design, combining dilated convolution and a dual-output structure, balances local texture realism and global spatiotemporal consistency, effectively improving the stability and discriminative sensitivity of adversarial training. To address the challenge of training deep networks for subtle motion manipulation, this invention redesigns the Patch GAN discriminator. By incorporating dilated convolution and a hybrid discriminator combining global and local adversarial losses, the network becomes more sensitive to micro-motions, stabilizing adversarial training and effectively preventing mode collapse.

[0088] In terms of performance, this invention demonstrates significant advantages across multiple evaluation metrics. Extensive experimental verification showed that at 20x magnification, the method achieved an average MUSIQ score of 46.44, ranking first in 8 out of 10 test sequences. Even under extreme magnification conditions of 40-80x, this method maintains stable visual quality with no significant artifacts, exhibiting excellent robustness to complex scenes and large-scale motion.

[0089] In terms of application value, this invention provides an effective tool for solving technical problems in multiple professional fields. In the field of industrial inspection, the method of this invention can amplify minute mechanical vibrations and deformations, providing intuitive evidence for equipment condition monitoring and fault diagnosis; in medical diagnosis, it can clearly present physiological micro-movements such as vascular pulsation and tissue tremors, assisting doctors in making accurate diagnoses; in scientific research, it provides strong technical support for the visualization and analysis of minute dynamic processes in scientific research activities such as physical experiments and biological observations; in the field of security monitoring, it can enhance subtle activities in monitoring videos, improving the accuracy and timeliness of security warnings.

[0090] Finally, it is particularly important to note that the closed-loop motion control framework proposed in this invention has broad technical versatility. This method is not limited to traditional RGB video motion amplification, but can be extended to various imaging modalities such as infrared, microscopy, and remote sensing, providing a unified technical solution for small motion analysis in multiple fields. This versatility gives this invention a wider application prospect and a longer technological lifespan, laying a solid foundation for subsequent technological evolution and application innovation.

[0091] To better illustrate the technical solution of the present invention, this embodiment is described in the following parts.

[0092] I. Generator

[0093] In this embodiment, the motion magnification module and the motion reduction module share a generator network, that is, the network structure is the same but the network parameters are different.

[0094] Let's take the motion amplification module as an example. Figure 1 As shown, the generator adopts an encoder-manipulator-decoder structure.

[0095] 1. Encoder

[0096] The encoder is used to extract multi-scale features from the reference frame and the query frame respectively, and obtain shallow features, medium features and deep features of the reference frame and the query frame.

[0097] The encoder consists of cascaded padding layers and three convolutional layers; each convolutional layer consists of cascaded convolutional modules and a ReLU activation function; the three convolutional layers are used to output shallow, medium, and deep features of the reference frame, respectively.

[0098] Specifically, the encoder module employs a three-tiered architecture for multi-scale feature extraction. Initially, two video frames (a reference frame and a query frame) are input into the encoder, and a 7×7 convolution with reflection padding is used to convert the RGB input into 32-channel features (Conv32_k7). Then, instance normalization and ReLU activation (IN ReLU) are applied to generate basic feature maps, denoted as ftA0 / ftB0. Subsequently, two stride-2 3×3 convolutions progressively downsample the features to 64 and 128 channels (Conv64_k3s2p1 and Conv128_k3s2p1), capturing increasingly abstract representations, denoted as ftA1 / ftB1 and ftA2 / ftB2. This design follows a pyramidal channel expansion pattern (3→32→64→128), where the feature depth doubles with each spatial downsampling step. Reflection padding ensures boundary consistency, while instance normalization and in-situ operations optimize memory usage. The resulting compressed representation retains the essential features while reducing memory consumption to 1 / 16 of the input size, providing an effective basis for subsequent operations.

[0099] 2. Control device

[0100] The manipulator is used to calculate the feature differences between the deep features of the reference frame and the query frame, and to weight these feature differences through a channel attention mechanism to obtain channel-enhanced features. After amplifying the channel-enhanced features through a motion magnification factor, they are fused with the deep features of the query frame to obtain the manipulated features.

[0101] The manipulator module facilitates cross-domain feature adaptation by selectively amplifying and propagating meaningful inter-frame differences. Its core operation involves computing a feature shift Δf = ftB² - ftA² between the encoded representations of two input frames. This shift is then processed by a channel attention mechanism, which generates channel-weighted coefficients through global feature aggregation and dimensionality reduction, thereby highlighting transform-related features while suppressing noise. We prioritize channel attention over spatial selection because it better preserves structural spatial relationships, achieves higher computational efficiency for high-resolution features, and aligns with observations that inter-domain differences are primarily reflected in feature channels rather than spatial locations. Finally, the attention-weighted shift is scaled by a given amplification factor α and combined with the feature ftB² of the query frame.

[0102] The calculation formula for the manipulator is expressed as follows:

[0103] M(X a ,X b ,α)=ftB2+α*Attention(ftB2-ftA2);

[0104] In the formula: M(·) represents the manipulator; Attention(·) represents the channel attention operation, used to emphasize the feature channels most relevant to motion transformation; ftA2 represents the reference frame X. a The deep features; ftB2 represents the query frame X b The deep features; α represents the motion amplification factor.

[0105] 3. Decoder

[0106] The decoder is used to progressively fuse the manipulated features with the shallow and mid-level feature maps of the query frame through a U-Net structure containing skip connections, thereby constructing a motion-amplified generated frame.

[0107] The decoder includes several cascaded enhanced residual blocks, a first upsampling module, a first convolutional layer, a second upsampling module, a second convolutional layer, a first padding layer, and a third convolutional layer; the output of the second convolutional layer is fused with the mid-level features of the query frame and then input to the first padding layer; the output of the third convolutional layer is fused with the shallow features of the query frame and then used as the output of the decoder to generate an amplified frame after motion amplification.

[0108] The enhanced residual block consists of a cascaded second padding layer, a fourth convolutional layer, a third padding layer, a fifth convolutional layer, and a ReLU activation layer; the output of the fourth convolutional layer is either Dropout processed or identity processed before being input to the second padding layer; the output of the fifth convolutional layer is added element-wise to the input of the enhanced residual block before being input to the ReLU activation layer.

[0109] The first, second, and fourth convolutional layers consist of cascaded convolutional modules and ReLU activation functions; the third convolutional layer consists of cascaded convolutional modules and Tanh activation functions; and the fifth convolutional layer consists of convolutional modules.

[0110] Specifically, the decoder module employs a multi-stage reconstruction pipeline, progressively restoring spatial resolution while preserving amplified motion features. Reconstruction begins with six enhanced residual blocks operating on 128 channels of features at a quarter-resolution. Each residual block contains two reflection-filled 3×3 convolutions with instance normalization and ReLU activation, directly fusing shallow features from the corresponding resolution level of the encoder via skip connections. These shallow features, rich in image texture and edge details, combined with the motion-rich deep features from the manipulator output, ensure that the reconstructed image maintains high spatial fidelity and clear details while amplifying motion, effectively avoiding blurring and distortion. The first two blocks further integrate dropout regularization (p=0.5) to prevent overfitting. These residual transformations stabilize the feature representation while preserving the amplified motion characteristics.

[0111] After residual processing, the decoder performs symmetric upsampling using the U-Net excitation architecture. The upsampling stage consists of two consecutive stages: the first upsampling operation (up1) doubles the spatial resolution (from 1 / 4 to 1 / 2 scale) and then performs a 3×3 convolution, reducing the channel depth from 128 to 64. The second upsampling operation (up2) uses corresponding convolutions from 64 to 32 to reduce the channels and restore the full spatial resolution (from 1 / 2 to the original scale). Crucially, each upsampling stage includes skip connections that fuse features from the corresponding encoder level: up1 combines with ftB1, and up2 integrates with ftB0. This multi-scale feature fusion ensures that high-frequency details of the original input are preserved during reconstruction. The final output layer (out) applies reflection-filled 7×7 convolutions (Conv3_k7) and Tanh activations to generate upscaled RGB frames.

[0112] II. Discriminator

[0113] In this embodiment, the first discriminator and the second discriminator are enhanced discriminators with the same structure, that is, the network structure is the same but the network parameters are different.

[0114] Let's take the motion amplification module as an example. To stabilize training and improve the ability to distinguish between micro-motions and spatiotemporal artifacts, we redesigned the Patch GAN discriminator from pix2pix, such as... Figure 2 As shown in (b).

[0115] 1. Backbone network and dilated convolution

[0116] Two cascaded dilated convolutional modules are connected after the last basic convolutional module in the Patch GAN backbone network. The output of the second dilated convolutional module is used as the output of the backbone network.

[0117] Specifically, the discriminator backbone consists of multiple convolutional layers. Unlike the standard Patch GAN, this embodiment introduces dilated convolutions in the intermediate layers, with dilation rates of 2 and 3. This exponentially expands the receptive field without increasing parameters or sacrificing resolution, enabling the discriminator to capture broader contextual information and long-range spatiotemporal dependencies, thereby more effectively identifying artifacts such as flickering and jitter caused by temporal inconsistencies.

[0118] 2. Dual-output structure

[0119] The backbone network's output is fed into two paths: the first path is the original local discrimination path of Patch GAN, which outputs an N×N matrix, where each element corresponds to the authenticity judgment of a local region of the input video frame; the second path applies global average pooling to the features output by the backbone network to compress the feature map into a feature vector, and then outputs a single global authenticity scalar based on the feature vector through a fully connected layer.

[0120] Specifically, the local discrimination path inherits from Patch GAN and outputs an N x N matrix (e.g., 30x30), where each element corresponds to the authenticity judgment of a local region of the input image, aiming to ensure the texture realism of the local region.

[0121] Global discrimination path: Based on the features extracted by the backbone network, global average pooling is applied to compress the feature map into a single feature vector, which is then output as a single global realism scalar through a fully connected layer. This path is used to evaluate the coherence of the entire image frame, the rationality of the overall structure, and the presence of frame-level anomalies.

[0122] This dual-output design, which combines local and global perspectives, allows the discriminator to supervise the generator's training from both micro and macro levels simultaneously.

[0123] III. Loss Function

[0124] The training process in this embodiment is based on the Cycle GAN framework, and its data flow is as follows: Figure 2 As shown in (a). The training objective is to optimize two generators G (motion amplification generator G). mag and motion shrink generator G min ) and two discriminators D(D mag Distinguish G mag The generated magnified frame, D min Distinguish G min (Generated scaled-down frame).

[0125] Training Data: We utilized the synthetic dataset proposed in the paper "Learning based Video Motion Magnification," which has been widely adopted by various deep learning-based VMM methods, and its effectiveness has been well-proven in existing literature. Some examples of this dataset are shown below. Figure 3 As shown.

[0126] Specifically, the fusion loss calculation module calculates the fusion loss using the following formula:

[0127] L total =λ adv *L adv +λ cyc *Lcyc +λ id *L id +λ pct *L pct ;

[0128] In the formula: L total L represents the fusion loss; adv L represents the sum of the discrimination loss of the first discriminator for generating the magnified frame and the discrimination loss of the second discriminator for generating the reference frame; cyc L represents the cycle consistency loss calculated based on the difference between the generated reference frame and the corresponding reference frame; id This represents the identity loss between the generated magnified frame and the actual magnified frame; L pct λ represents the perceptual loss between the generated magnified frame and the actual magnified frame; adv , λ cyc , λ id , λ pct This represents the weighting coefficient for each loss.

[0129] Optimizer and Hyperparameters: The Adam optimizer is used, with an initial learning rate (lr) of 0.0002 and momentum parameters β1 = 0.5 and β2 = 0.999. The batch size is set to 4 based on GPU memory. Training cycles are typically 100-200 epochs.

[0130] 1. Combating Losses

[0131] Combat loss L adv The generator is driven to produce amplified frames that are sufficiently realistic to be mistaken for genuine ones. The adversarial loss derives from the discriminator's evaluation of the generated frames. While traditional adversarial losses rely on a single output of the discriminator, our redesigned architecture employs a dual-output discriminator. Therefore, the adversarial loss here consists of a global output and a patched output.

[0132] Combat loss L adv The calculation formula is:

[0133]

[0134] Where: G mag (·) indicates the motion amplification module; X a Indicates a reference frame; X b This represents the query frame; α represents the motion magnification factor; Y represents the generated magnified frame; L represents the actual magnified frame. global Indicates the global discrimination path output of the discriminator; L patch This indicates the local path output of the discriminator.

[0135] Adversarial loss is used to make the video frames generated by the motion magnification and motion downscaling modules approximate the distribution of real video frames. This design forces the network to pay more attention to local details in each patch, thereby significantly improving the fine-grained quality of the magnified output.

[0136] 2. Cyclic consistency loss

[0137] Cyclic consistency loss L cyc As a key component of our bidirectional loop, the cycle consistency loss aims to encourage the reversibility and realism of motion. It seeks to alleviate a core challenge in VMMs: amplifying subtle motions while minimizing unrealistic distortions.

[0138] Cyclic consistency loss L cyc The calculation formula is:

[0139]

[0140] In the formula: L1 represents the L1 norm; X a Indicates a reference frame; Indicates the generation of a reference frame; G mag (·) indicates the motion amplification module; G min (·) indicates the motion reduction module; X b Y represents the query frame; Y represents the actual magnified frame. This indicates the generation of a magnified frame; α represents the motion magnification factor.

[0141] Cyclic consistency loss is used to constrain the differences between the video frames reconstructed by the motion downscaling module and the original reference frame. This loss encourages the video frames, after being upscaled and subsequently downscaled, to be reconstructed to be close to their original versions. This round-trip constraint acts as a form of self-supervision, guiding the generator to learn more reversible transformations. In practice, this formulation helps reduce common artifacts such as temporal flicker and spatial distortion because it discourages the network from producing upscaling that deviates significantly from reversible cycles. Therefore, this loss contributes to achieving higher visual fidelity and more reliable motion upscaling, especially at higher magnifications.

[0142] 3. Loss of identity L id

[0143] Identity loss L id The identity loss is used as a regularization term to stabilize training. It is based on the concept that if the generator receives input that is already part of the target domain, ideally it should return the input unchanged. We apply this by feeding the reconstructed magnified frame, the real magnified frame, and the downscaling factor to the downscaling generator, and calculating the L1 difference between its output and the query frame.

[0144] Identity loss L id The calculation formula is:

[0145]

[0146] In the formula: L1 represents the L1 norm; G min (·) indicates the motion reduction module; X b Indicates a query frame; This indicates the generation of a magnified frame.

[0147] Identity loss is used to prevent the generated frames from changing too drastically from the original query frames, thus preserving overall information. This loss prevents the network from learning to distort frames that have been appropriately amplified, thereby helping to retain original content information and leading to more natural results.

[0148] 4. Perceived loss

[0149] Perceived loss L pct The purpose of introducing perceptual loss is to improve the visual quality of the generated frames, surpassing the performance achievable with pixel-based loss. It uses the LPIPS metric, which calculates the distance between the generated and target images in the feature space of a pre-trained deep network (such as VGG).

[0150] Perceived loss L pct The calculation formula is:

[0151]

[0152] In the formula: L2 represents the L2 norm; Y represents the generated magnified frame; L represents the actual magnified frame. LPIPS Learn to perceive image patch similarity loss.

[0153] The perceptual loss uses a pre-trained deep neural network to extract features and calculates the distance between the generated frame and the real frame in the feature space.

[0154] IV. Experimental Instructions

[0155] To verify the effectiveness of our method, we tested it on multiple publicly available datasets and self-collected real-world videos, and conducted qualitative and quantitative comparisons with seven state-of-the-art methods. Specifically, we curated a representative test dataset from publicly available real-world videos released for VMM tasks. Selection criteria prioritized motion complexity, scene diversity, and data quality, resulting in nine different motion scenarios (including human activity, structural vibration, etc.) to ensure comprehensive evaluation under diverse real-world conditions.

[0156] Quantitative Evaluation: We employ the Multi-Scale Image Quality Transformer (MUSIQ), a Transformer-based no-reference image quality assessment metric, as our evaluation standard. As a perceptual image quality assessment model, a higher MUSIQ score indicates better visual fidelity. To comprehensively evaluate our proposed Cycle GAN-based VMM method, we conducted comparative experiments with a set of established methods, including traditional methods, deep learning-based methods, and our own. Performance was evaluated using MUSIQ scores across different video sequences and multiple magnifications. Official implementations of all methods are available from their respective GitHub repositories. For fair evaluation, the linear and phase-based methods were configured to their maximum sampling constraint frequency range.

[0157] We first conducted a comprehensive quantitative comparison with seven state-of-the-art methods on ten different video sequences at 20x magnification. Table 1 presents the detailed quantitative results. For a more intuitive comparison of performance, Figure 4 MUSIQ scores for each sequence were plotted at 20x magnification. The red and green reference lines represent the scores of our method and the average scores of all other methods, respectively. The results show that our method consistently outperforms the baseline average across all sequences, achieving the highest scores in eight out of ten videos, only slightly lagging behind in the "baby" and "cattoy" sequences. Figure 5 As shown, our method ranks first with an average MUSIQ score of 46.4358, which is 5.16% higher than the second-ranked linear method (44.1571) and a significant 55.36% higher than the latest competitor EulerMormer (29.8883).

[0158] Table 1. MUSIQ scores for various methods at 20x magnification

[0159]

[0160] Linear methods maintain competitive quantitative metrics (second highest average MUSIQ) through lossless Laplacian pyramid decomposition, particularly adept at preserving low-frequency components and image structure. However, linear interpolation between pyramid levels introduces discontinuous transitions and edge artifacts, leading to a significant deterioration in visual quality. This disconnect between metrics and perceived quality will be visually presented in the qualitative analysis section. In contrast, learning-based methods exhibit significant performance fluctuations across different content types, while our method establishes a unique, balanced performance spectrum, providing consistently high-quality results across almost all sequences.

[0161] To assess robustness under more challenging conditions, we further extended the analysis to high-magnification scenarios such as 40×, 60×, and 80×. Figure 6The performance of each method at different magnifications was illustrated using a multi-subgraph layout. It is evident that the MUSIQ scores of all methods generally decrease as the magnification increases—reflecting the inherent difficulty of maintaining visual quality under strong motion magnification. Nevertheless, our method demonstrates outstanding robustness: achieving the highest MUSIQ values ​​at all magnifications in seven out of ten sequences; and consistently ranking among the top three in the remaining three sequences. Figure 7 The aggregated results further confirm that our method maintains the highest average MUSIQ score at every magnification level. Notably, the advantages of our method become more pronounced as the magnification increases, highlighting its enhanced ability to handle large motion scenes. These findings demonstrate that our method not only provides superior perceptual quality under normal magnification conditions but also maintains stable high performance under extreme conditions, fully demonstrating its strong adaptability in practical applications.

[0162] The effectiveness of our proposed Cycle GAN-based training framework is... Figure 8 Quantitative validation was achieved. Direct comparisons show that the generator trained within our Cycle GAN framework consistently outperforms generators trained directly with motion amplification in 90% of video sequences. This comprehensive improvement in MUSIQ scores demonstrates that the adversarial and cycle consistency constraints provided by the Cycle GAN framework are crucial for guiding the generator to produce higher perceptual quality results.

[0163] Qualitative assessment: Figure 9 This paper presents a comparison of the various methods' performance at multiple magnification levels, from 20x to 80x, on frame 113 of the "balloon" sequence (capturing the complex motion of a balloon bursting). At 20x magnification, the linear method and LBVMM exhibit severe distortion, only able to identify the balloon's approximate location; the phase method preserves the outline but loses surface wrinkles and water splash textures; the acceleration method and EulerMormer avoid structural distortion but exhibit significant blurring; MDVMM and STBVMM perform relatively better, but water splash details remain blurry. In contrast, this proposed method demonstrates the best sharpness and detail integrity in both the balloon outline and the water splash area.

[0164] Figure 10This advantage was further validated by frame 26 of a "bird" sequence (recording a wooden balancing bird swaying in a breeze) captured by a real mobile phone. As magnification increased, most contrast methods exhibited increased distortion and blurriness, while our method maintained a stable and clear output throughout the magnification process. Specifically: linear methods maintained structural recognizability but suffered significant visual distortion; phase methods performed well in terms of structural integrity but had poor visual quality; acceleration methods produced severe distortion with increasing magnification; all deep learning contrast methods performed poorly in this real-world video, exhibiting obvious blurring and distortion, with some even failing to maintain the basic structure of the target. While our method was slightly inferior to linear methods in contour sharpness, it achieved a better balance in visual fidelity, structural preservation, and motion coherence.

[0165] Analysis of multiple test videos revealed distinct characteristics for each method: traditional linear and phase methods, due to their preservation of more structural information from the original image, exhibited significant advantages in contour preservation, but their visual appearance was unnatural and texture detail recovery was limited; the acceleration method performed well at low magnification, but significant artifacts and blurring occurred as the magnification increased; LBVMM, as an initial attempt at deep learning, showed obvious distortion, but its performance was less affected by magnification; MDVMM achieved balanced performance in most sequences through time-frequency domain information fusion, but its overall sharpness remained insufficient; STBVMM and EulerMormer suffered from performance degradation and instability at high magnification, respectively. While our proposed method did not surpass traditional methods in contour sharpness, it outperformed most deep learning methods in the vast majority of cases, achieving the optimal balance between overall sharpness and high-magnification robustness. It should be noted that our method exhibited color cast compared to the original video in all test sequences, the specific causes of which require further investigation.

[0166] In summary, this invention effectively addresses the core challenges of motion amplification under small sample sizes by employing a closed-loop motion control framework, an attention-driven feature manipulator, an enhanced discriminator, and joint optimization of multi-target loss. It achieves high-fidelity and robust motion amplification, and has broad application prospects in fields such as industrial inspection and medical analysis.

[0167] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit the technical solutions. Those skilled in the art should understand that any modifications or equivalent substitutions to the technical solutions of the present invention without departing from the spirit and scope of the present invention should be covered within the scope of the claims of the present invention.

Claims

1. A video motion amplification method based on recurrent generative adversarial networks, characterized in that, include: S1: Obtain the video sequence data to be processed, and determine the reference frame and query frame in the video sequence data; S2: Input the reference frame, query frame and motion magnification factor into the trained video motion magnification model for motion magnification processing, and output the generated magnified frame after motion magnification. The processing steps for training a video motion magnification model include: S201: Obtain the reference frame, query frame, motion magnification factor and motion reduction factor used as training data, as well as the real magnified frame corresponding to the query frame as the training input for the video motion magnification model; S202: The motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame after motion magnification; S203: The motion reduction module reduces the feature differences between the generated magnified frame and the query frame based on the motion reduction factor, and fuses them with the features of the query frame to construct a generated reference frame corresponding to the reference frame. S204: The first discriminator determines the difference between the generated magnified frame and the corresponding real magnified frame; S205: The second discriminator determines the difference between the generated reference frame and the corresponding reference frame; S206: The fusion loss calculation module calculates the fusion loss based on the discrimination results of the first discriminator and the second discriminator, combined with the difference between the generated magnified frame and the corresponding real magnified frame and the difference between the generated reference frame and the corresponding reference frame, and optimizes the network parameters of the video motion magnification model based on the fusion loss. S207: Repeat steps S201 to S206 to iteratively train the video motion magnification model until the model converges or reaches the maximum number of iterations. After the video motion magnification model is trained, the motion magnification module amplifies the feature differences between the reference frame and the query frame based on the motion magnification factor, and fuses them with the features of the query frame to construct the generated magnified frame as the output of the video motion magnification model.

2. The video motion amplification method based on recurrent generative adversarial networks as described in claim 1, characterized in that: In step S202, the motion amplification module is a generator network that includes cascaded encoders, manipulators, and decoders; The encoder is used to extract multi-scale features from the reference frame and the query frame respectively, and obtain shallow features, medium features and deep features of the reference frame and the query frame. The manipulator is used to calculate the feature differences between the deep features of the reference frame and the query frame, and to weight these feature differences through a channel attention mechanism to obtain channel-enhanced features. After amplifying the channel enhancement features using a motion amplification factor, they are then fused with the deep features of the query frame to obtain the manipulated features. The decoder is used to fuse the manipulated features with the shallow and mid-level feature maps of the query frame step by step to construct the generated magnified frame after motion magnification.

3. The video motion amplification method based on recurrent generative adversarial networks as described in claim 2, characterized in that: In step S202, the encoder includes cascaded padding layers and three convolutional layers; the convolutional layers include cascaded convolutional modules and ReLU activation functions; The three convolutional layers are used to output shallow, medium, and deep features of the reference frame, respectively.

4. The video motion amplification method based on recurrent generative adversarial networks as described in claim 2, characterized in that: In step S202, the calculation formula for the manipulator is expressed as follows: M(X a ,X b ,α)=ftB2+α*Attention(ftB2-ftA2); In the formula: M(·) represents the manipulator; Attention(·) represents the channel attention operation, used to emphasize the feature channels most relevant to motion transformation; ftA2 represents the reference frame X. a The deep features; ftB2 represents the query frame X b The deep features; α represents the motion amplification factor.

5. The video motion amplification method based on recurrent generative adversarial networks as described in claim 2, characterized in that: In step S202, the decoder includes several cascaded enhanced residual blocks, a first upsampling module, a first convolutional layer, a second upsampling module, a second convolutional layer, a first padding layer, and a third convolutional layer; the output of the second convolutional layer is fused with the mid-level features of the query frame and then input to the first padding layer; the output of the third convolutional layer is fused with the shallow features of the query frame and then used as the generated magnified frame after motion amplification of the decoder output. The enhanced residual block consists of a cascaded second padding layer, a fourth convolutional layer, a third padding layer, a fifth convolutional layer, and a ReLU activation layer; the output of the fourth convolutional layer is either Dropout processed or identity processed before being input to the second padding layer; the output of the fifth convolutional layer is added element-wise to the input of the enhanced residual block before being input to the ReLU activation layer. The first, second, and fourth convolutional layers consist of cascaded convolutional modules and ReLU activation functions; the third convolutional layer consists of cascaded convolutional modules and Tanh activation functions; and the fifth convolutional layer consists of convolutional modules.

6. The video motion amplification method based on recurrent generative adversarial networks as described in claim 1, characterized in that: In step S204, the first discriminator is generated based on the improved Patch GAN: Two cascaded dilated convolutional modules are connected after the last basic convolutional module of the Patch GAN backbone network, and the output of the second dilated convolutional module is used as the output of the backbone network. The backbone network's output is fed into two paths: the first path is the original local discrimination path of Patch GAN, which outputs an N×N matrix, where each element corresponds to the authenticity judgment of a local region of the input video frame; the second path applies global average pooling to the features output by the backbone network to compress the feature map into a feature vector, and then outputs a single global authenticity scalar based on the feature vector through a fully connected layer.

7. The video motion amplification method based on recurrent generative adversarial networks as described in claim 1, characterized in that: In step S206, the fusion loss calculation module calculates the fusion loss using the following formula: L total =λ adv *L adv +λ cyc *L cyc +λ id *L id +λ pct *L pct ; In the formula: L total L represents the fusion loss; adv L represents the sum of the discrimination loss of the first discriminator for generating the magnified frame and the discrimination loss of the second discriminator for generating the reference frame; cyc L represents the cycle consistency loss calculated based on the difference between the generated reference frame and the corresponding reference frame; id This represents the identity loss between the generated magnified frame and the actual magnified frame; L pct λ represents the perceptual loss between the generated magnified frame and the actual magnified frame; adv , λ cyc , λ id , λ pct This represents the weighting coefficient for each loss.

8. The video motion amplification method based on recurrent generative adversarial networks as described in claim 7, characterized in that: Combat loss L adv The calculation formula is: Where: G mag (·) indicates the motion amplification module; X a Indicates a reference frame; X b This represents the query frame; α represents the motion magnification factor; Y represents the generated magnified frame; L represents the actual magnified frame. global Indicates the global discrimination path output of the discriminator; L patch This indicates the local path output of the discriminator.

9. The video motion amplification method based on recurrent generative adversarial networks as described in claim 7, characterized in that: Cyclic consistency loss L cyc The calculation formula is: In the formula: L1 represents the L1 norm; X a Indicates a reference frame; Indicates the generation of a reference frame; G mag (·) indicates the motion amplification module; G min (·) indicates the motion reduction module; X b Indicates a query frame; Y represents the actual magnified frame; This indicates the generation of a magnified frame; α represents the motion magnification factor.

10. The video motion amplification method based on recurrent generative adversarial networks as described in claim 7, characterized in that: Identity loss L id The calculation formula is: In the formula: L1 represents the L1 norm; G min (·) indicates the motion reduction module; X b Indicates a query frame; This indicates the generation of a magnified frame; Perceived loss L pct The calculation formula is: In the formula: L2 represents the L2 norm; Y represents the generated magnified frame; L represents the actual magnified frame. LPIPS Learn to perceive image patch similarity loss.

Images