Video Subtitle Removal via Frame Correlation Inpainting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image inpainting techniques struggle to effectively remove integrated subtitles from videos without damaging the original content, especially when the original video is not available, leading to difficulties in editing or removing them without obscuring important parts of the screen.
Innovation Solution
An image inpainting method and device that select target and reference frames to perform inter-frame and intra-frame correlation-based inpainting, using a frame selection unit and inpainting unit to generate high-quality inpainted images by referencing neighboring pixel information and scene transition data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If subtitles are integrated with the video, then the video becomes more visually complete, but the subtitles become difficult to edit or remove without damaging the original content
Solution Approach 1:
The patent segments the video processing into multiple frames and identifies specific subtitle regions within each frame. By dividing the video into individual frames and locating subtitle areas separately, the system can selectively remove subtitles while preserving the rest of the video content, thus enabling easy subtitle removal without damaging the original video content.
Solution Approach 2:
The patent uses an intermediary process of analyzing frame correlations and identifying subtitle-specific regions as a mediator between the integrated video and the final output. This intermediary analysis allows the system to distinguish subtitle content from video content, enabling precise removal while maintaining content integrity.
2Manufacturing precision
If existing inpainting techniques are used to remove subtitles, then the obscured area can be filled, but the quality of the inpainted image deteriorates
Solution Approach 1:
The patent applies dynamic frame correlation analysis to adaptively select reference frames based on their correlation with the target frame. By dynamically adjusting which frames are used as references for inpainting, the system maintains high image quality while removing subtitles, avoiding the degradation that occurs with static or generic inpainting methods.
Solution Approach 2:
The patent incorporates feedback mechanisms by analyzing the correlation between frames and using this information to guide the inpainting process. The system evaluates frame correlations and uses this feedback to select the most appropriate reference frames, ensuring high-quality inpainting results that preserve image fidelity.
3Quantity of substance
If the original video is not available, then cost savings are achieved, but the ability to recover original content without subtitles is compromised
Solution Approach 1:
The patent performs preliminary analysis of frame correlations and identifies subtitle regions before actual removal takes place. By preparing reference frames and analyzing correlations in advance, the system can recover original content quality without needing to store the original video, achieving both cost savings and content recovery.
Solution Approach 2:
The patent creates copies of relevant frame information and uses these copies as reference material for inpainting. By copying and storing only the necessary reference frame data rather than the entire original video, the system achieves cost-effective storage while maintaining the ability to recover original content quality through the copied reference information.
Data Source
AI summary
The present disclosure provides an image inpainting method and an image inpainting device. The present disclosure in at least one embodiment provides a method of inpainting an image area obscured by an object included in an image, including selecting one or more frames among input frames of the image, wherein each of the one or more frames is selected as a target frame including an inpainting area obscured by the object or as a reference frame including information on the inpainting area, and performing at least one of inter inpainting or intra inpainting depending on a number of frames selected as the reference frame.