Image Processing for AR Effect Tracking Through Optical Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing technologies fail to maintain the relative position of augmented reality (AR) effect models consistently with other objects in an AR scene when the view angle changes, leading to a subpar user experience.
Innovation Solution
An image processing method and apparatus that determines the target display position of AR effect models based on current and historical video frames, using optical flow field information and transformation matrices to adjust the model's position accordingly, ensuring it remains relative to the object despite changes in the view angle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the AR effect model remains static in the display interface, then the model maintains its original position, but the model fails to respond to view angle changes and does not maintain consistent relative position with other objects in the AR scene
Solution Approach 1:
The patent applies the dynamics principle by making the AR effect model's display position adaptive rather than static. The system dynamically adjusts the model's position in response to view angle changes by calculating transformation matrices from optical flow fields between consecutive video frames. This allows the model to maintain consistent relative positioning with target objects while adapting to changing camera angles, resolving the contradiction between stability and adaptability.
2Reliability
If the AR effect model adapts to view angle changes by updating its position, then the model maintains consistent relative position with objects, but the computational complexity increases due to optical flow field calculation and transformation matrix determination
Solution Approach 1:
The patent applies partial action by calculating optical flow fields and transformation matrices only for key points between consecutive video frames rather than processing the entire image. This selective approach focuses computational resources on critical position tracking while reducing overall processing complexity, enabling the system to maintain reliable relative positioning without excessive computational burden.
3Speed
If the model position is updated in every video frame, then the model responds smoothly to view angle changes, but the processing time and computational resources increase
Solution Approach 1:
The patent implements continuous useful action by updating the AR effect model's position in every video frame based on calculated transformation matrices. This continuous updating ensures smooth real-time response to view angle changes while maintaining efficient processing through optimized optical flow calculations, achieving both high response speed and reasonable processing time.
Data Source
AI summary
Embodiments of the disclosure provide an image processing method and apparatus, an electronic device and a storage medium. The method includes: when it is detected that a main body to be processed on a display interface is a target main body, adding an augmented reality (AR) special effect model for the target main body; according to a current video frame and a previous historical video frame, determining a target display location of the AR special effect model in the current video frame; displaying the AR special effect model at the target display position to obtain a target special effect video frame corresponding to the current video frame.