Neural Network Training for Video Frame Time Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional neural network model training methods for image processing fail to consider time consistency between video frames, resulting in significant flickering during feature conversion, which negatively impacts the feature conversion effect.
Innovation Solution
The method involves training a neural network model using time loss and feature loss as feedback and adjustment bases, where optical flow information is used to adjust intermediate images, ensuring time consistency between adjacent video frames, and combining processor capabilities for improved calculation speed without compromising feature conversion quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional neural network model training methods are used for image processing, then the training process is simple and fast, but time consistency between video frames is not considered resulting in significant flickering during feature conversion
Solution Approach 1:
The patent introduces a feedback mechanism by calculating optical flow between adjacent video frames and using this information to generate time loss. This time loss is then fed back into the training process to adjust the neural network model, ensuring that the model learns to maintain temporal consistency. The feedback loop includes: obtaining video frames, calculating optical flow, computing time loss based on differences between frames, and updating the model parameters accordingly.
Solution Approach 2:
The patent segments the training process into distinct modules: feature extraction module, optical flow calculation module, time loss calculation module, and model training module. This segmentation allows each component to be optimized independently and makes the overall complex system more manageable. The segmentation separates the computation of spatial features from temporal consistency checks.
2Reliability
If optical flow information is used to adjust intermediate images for time consistency, then flickering during feature conversion is reduced, but processing time and computational complexity increase
Solution Approach 1:
The patent performs preliminary action by pre-calculating optical flow information between adjacent video frames during the training process. This optical flow data is stored and reused during inference, avoiding the need to recalculate optical flow for every new video frame processed. The preliminary computation of optical flow metrics during training enables faster processing during actual feature conversion tasks.
Solution Approach 2:
The patent changes the training parameters by introducing time loss as an additional objective function alongside traditional feature loss. The model is trained to minimize both spatial feature differences and temporal inconsistencies. By adjusting the loss function parameters and training objectives, the system learns to produce intermediate images that satisfy both feature conversion requirements and temporal consistency constraints.
3Manufacturing precision
If time loss and feature loss are combined for training, then feature conversion effect is enhanced, but training cost and computational resources increase
Solution Approach 1:
The patent changes the training parameters by introducing time loss as an additional objective function alongside traditional feature loss. The model is trained to minimize both spatial feature differences and temporal inconsistencies. By adjusting the loss function parameters and training objectives, the system learns to produce intermediate images that satisfy both feature conversion requirements and temporal consistency constraints.
Solution Approach 2:
The patent ensures continuity of useful action by maintaining the optical flow information and time loss calculations throughout the training process. Rather than performing these computations only at the end, the system continuously uses optical flow data to guide feature extraction and intermediate image generation. This continuous feedback ensures that temporal consistency is maintained at every training step, improving overall feature conversion precision.
Data Source
AI summary
A method, apparatus, and storage medium for training a neural network model used for image processing are described. The method includes: obtaining a plurality of video frames; inputting the plurality of video frames through a neural network model so that the neural network model outputs intermediate images; obtaining optical flow information between an early video frame and a later video frame; modifying an intermediate image corresponding to the early video frame according to the optical flow information to obtain an expected-intermediate image; determining a time loss between an intermediate image corresponding to the later video frame and the expected-intermediate image; determining a feature loss between the intermediate images and a target feature image; and training the neural network model according to the time loss and the feature loss, and returning to obtaining a plurality of video frames continue training until the neural network model satisfies a training finishing condition.