Multi-Layer Mesh Image Warping for Complex Edge Continuity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video frame interpolation technologies cause local image distortion due to image warping operations, especially in regions with complex geometric edges, leading to a noticeable difference between predicted and real frames, which degrades user visual experience.
Innovation Solution
An image processing method that identifies discontinuous image regions, constructs multiple mesh layers based on reconstruction attributes, and performs image warping on each layer using corresponding motion vectors to ensure continuous pixel regions, thereby minimizing distortion and ensuring image continuity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If image warping operation is performed to ensure image continuity, then image continuity is improved, but local image distortion occurs in regions with complex geometric edges
Solution Approach 1:
The patent divides the image into multiple mesh layers (first mesh layer, second mesh layer, etc.) based on depth information and pixel attributes. Each mesh layer is processed independently with its own motion vectors, allowing different regions to be warped according to their specific characteristics. This segmentation prevents distortion in complex geometric edge regions while maintaining overall image continuity.
Solution Approach 2:
The patent applies different warping strategies to different mesh layers. Static pixels in deeper layers are warped using motion vectors from adjacent frames, while dynamic pixels in front layers are handled differently. This local differentiation ensures that each region receives the appropriate warping treatment, reducing distortion in complex regions while maintaining continuity in simpler regions.
2Manufacturing precision
If multiple mesh layers are constructed to reduce distortion, then local image distortion is reduced, but processing complexity increases
Solution Approach 1:
The patent dynamically determines the number of mesh layers and their configurations based on the specific image content and depth information. The system adapts the mesh layer structure to the actual scene requirements, creating more layers only when necessary to handle complex geometric edges, thereby reducing overall processing complexity while maintaining distortion reduction benefits where needed.
Data Source
Figure 1
Figure 2
Figure 3(1)~3(4)
AI summary
This application provides an image processing method and apparatus, and an electronic device, and belongs to the field of smart terminal technologies. The method includes: creating at least one second mesh layer if a real frame image includes a reconstructed region satisfying a first preset condition and/or a second preset condition, and drawing pixels of the reconstructed region at a first mesh layer and the at least one second mesh layer respectively based on a reconstruction attribute of the reconstructed region. In this way, regardless of how complex a geometric edge of a discontinuous image region is, pixels having different attributes in the discontinuous image region can be respectively drawn at different mesh layers in a mesh reconstruction manner, to ensure that a pixel region in each mesh is a continuous pixel region. Therefore, when image warping processing is further performed on a mesh with continuous depths, a mesh pattern is not severely deformed, thereby ensuring image continuity, resolving a problem of distortion of the discontinuous image region having the complex geometric edge, and improving game experience of a user.