Video Texture Migration Using Reference-Frame Fusion to Prevent Flicker
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing video texture migration techniques suffer from inter-frame flicker due to random noises in the structural feature information, which affects the display quality of the generated texture migration video.
Innovation Solution
A method involving feature extraction on target frames of an initial video to generate first feature information, followed by feature fusion with reference frame information to obtain second feature information with reduced noise, and then combining this with texture feature information to generate a texture migration video, using a decoder in a video texture migration model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If feature extraction is performed on each video frame independently to generate texture migration images, then the processing speed is fast and the method is simple, but random noises are introduced causing inter-frame flicker
Solution Approach 1:
The patent applies preliminary action by extracting and storing structural feature information from reference frames before processing target frames. The encoder pre-processes reference frames to obtain structural features, which are then used as stable references during subsequent texture migration operations, avoiding the need to re-extract features for each frame and eliminating flicker caused by independent extraction.
Solution Approach 2:
The patent uses copying by replicating the structural feature information from reference frames and using it as a stable template for multiple target frames. Instead of independently extracting features from each frame, the system copies the pre-extracted structural features and applies them consistently across multiple frames, ensuring temporal stability and eliminating flicker effects.
2Device complexity
If feature fusion is performed using only current frame information, then the processing is simple and fast, but the generated video exhibits inter-frame flicker
Solution Approach 1:
The patent applies merging by combining structural feature information from multiple reference frames with texture feature information from target frames. The feature fusion module integrates these different types of features, weighting the stable structural features from reference frames against the current frame's texture features, thereby producing stable output that eliminates flicker while maintaining processing efficiency.
Solution Approach 2:
The patent uses an intermediary approach by introducing structural feature information as a mediating element between reference frames and target frames. This structural feature information acts as a stable intermediary that guides the texture migration process, ensuring consistency across frames and eliminating flicker without requiring complex processing of every frame.
3Use of energy by moving object
If independent frame processing is used, then computational resources are saved, but noise removal effectiveness is reduced
Solution Approach 1:
The patent applies preliminary action by performing noise removal and structural feature extraction on reference frames before they are used for texture migration. The encoder pre-processes reference frames to remove noise and extract stable structural features, which are then reused for multiple target frames. This approach removes noise effectively while saving computational energy, as the noisy reference frames are processed only once rather than repeatedly for each target frame.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
Provided in the embodiments of the present disclosure are a video texture migration method and apparatus, an electronic device and a storage medium. The method comprises: acquiring an initial video; performing feature extraction on a target frame in the initial video, so as to generate first feature information of the target frame, wherein the target frame is a video frame after an Nth video frame in the initial video, and the first feature information represents an image structure contour of the target frame; performing feature fusion on the first feature information of the target frame and reference feature information, so as to obtain second feature information of the target frame, wherein the reference feature information represents an image structure contour of a reference frame, the reference frame is a video frame before the target frame, and the second feature information is first feature information with random noise removed; and generating a texture migration video according to the second feature information of the target frame and corresponding texture feature information, wherein the texture feature information represents image texture details of a reference image. By means of the method, the problem of inter-frame flicker is avoided, and the display effect of the texture migration video is improved.