Video Object Removal via Local Frame Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video processing techniques are inefficient in removing objects or completing missing areas in video frames, particularly on mobile devices, due to low efficiency in object removal or missing area completion.
Innovation Solution
A method involving an electronic apparatus that determines a local frame sequence from a video, obtains a feature map sequence through encoding, calculates a feature flow sequence based on the feature map and a mask image, performs feature fusion between adjacent feature maps, and decodes the updated feature map to obtain a processed local frame sequence, while also selecting a reference frame sequence based on similarity for enhanced processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional object removal or missing area completion techniques are used in video frames, then the processing can be completed, but the processing efficiency is low
Solution Approach 1:
The patent divides the video processing into local frame sequences and selects reference frames from candidate frames. By segmenting the processing task into smaller units (local frames with specific candidate reference frames) rather than processing the entire video sequentially, the system achieves parallel processing capabilities and improves overall efficiency while reducing processing time.
2Manufacturing precision
If feature fusion between adjacent feature maps is performed, then the quality of object removal and missing area completion is improved, but the computational complexity increases
Solution Approach 1:
The patent performs encoding to obtain feature maps before the actual object removal or missing area completion process. By preparing the feature maps in advance and selecting reference frames beforehand, the system reduces the computational burden during the main processing stage. This preliminary preparation allows for higher quality results through feature fusion without proportionally increasing overall computational complexity.
3Measurement precision
If reference frame selection based on similarity is implemented, then the accuracy of inpainting processing is improved, but the processing time for frame selection increases
Solution Approach 1:
The patent calculates similarity between candidate frames and current frames to select reference frames, but applies this selectively rather than uniformly across all frames. By focusing computational effort on identifying suitable reference frames only where needed (based on local frame characteristics and similarity metrics), the system achieves high inpainting accuracy without proportionally increasing frame selection time across the entire video.
Data Source
AI summary
A method performed by an electronic apparatus, includes: determining a local frame sequence from a video; obtaining a feature map sequence of the local frame sequence by encoding the local frame sequence; determining a feature flow sequence of the local frame sequence based on the feature map sequence of the local frame sequence and a mask image sequence regarding an object to be removed, the mask image sequence being corresponding to the feature map sequence; obtaining an updated feature map sequence of the local frame sequence, by performing, based on the feature flow sequence, feature fusion between adjacent feature maps in the feature map sequence; and obtaining a processed local frame sequence, by decoding the updated feature map sequence of the local frame sequence.