AR Image Post-Processing Using Stencil Buffer Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing image processing methods in augmented reality (AR) scenarios fail to provide a flexible and immersive user experience by performing post-processing only on content rendered by a virtual camera or user-assigned objects, leading to potential frame rate issues and video lags.
Innovation Solution
The method employs a stencil buffer to differentiate pixels to be post-processed from those not to be, updating stencil values during rendering, allowing selective post-processing on the RGB image, thereby reducing the need for additional virtual cameras and pixel mask images, and enhancing user immersion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If post-processing is performed on all pixels in the RGB image, then complete image processing is achieved, but processing time increases and frame rate decreases
Solution Approach 1:
The patent segments the image processing by dividing pixels into two categories: pixels to be post-processed (virtual camera rendered pixels) and pixels not to be post-processed (physical camera captured pixels). The stencil buffer is used to mark and differentiate these pixel segments, allowing selective post-processing only on the virtual camera rendered content, thereby reducing overall processing time and improving frame rate while maintaining completeness of necessary post-processing operations.
2Productivity
If post-processing is performed only on virtual camera content or user-assigned objects, then processing time is reduced and frame rate improves, but user immersion and viewing experience deteriorate
Solution Approach 1:
The patent applies local quality by performing post-processing selectively on specific regions (virtual camera rendered pixels) rather than uniformly on the entire image. The stencil buffer enables precise control over which local regions undergo post-processing, allowing the system to enhance virtual content with visual effects while leaving real-world captured content unchanged, thereby maintaining user immersion and viewing experience while improving frame rate.
3Ease of operation
If additional virtual cameras and pixel mask images are used for selective post-processing, then processing control is improved, but device complexity increases
Solution Approach 1:
The patent applies universality by making the stencil buffer serve multiple functions: it marks pixels to be post-processed, differentiates between virtual and physical camera content, and controls the post-processing operation. This multi-functional approach eliminates the need for separate pixel mask images and additional virtual cameras, achieving precise processing control while reducing device complexity.
Data Source
AI summary
The present disclosure provides an image processing method and apparatus. The method comprises: obtaining an RGB image and a stencil buffer, wherein the RGB image comprises image information taken by a physical camera and image information rendered by a virtual object rendering module, and the stencil buffer is used for storing stencil values corresponding to pixels on the RGB image; obtaining a target image by processing the RGB image according to the stencil buffer; obtaining a post-processed image by post-processing the target image; determining an image to be displayed according to the post-processed image. In this way, the post-processing object is flexible and controllable. The user therefore has a stronger sense of participation and a better experience.