AR Image Display Method for Free Perspective Video Viewing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for displaying free perspective videos provide limited user freedom in viewing, as they typically rely on separate interactive players with slider bars, resulting in a poor user experience.

Innovation Solution

An image display method that converts pixel points from a foreground image in a video frame into an augmented reality (AR) coordinate system, combines this with a background image captured by a background capturing device, and displays the result as an augmented reality image, allowing users to interactively view videos with six degrees of freedom.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a separate interactive player with slider bar is used to display free perspective video, then the video can be played with different perspectives, but the user viewing freedom is limited and the user experience is poor

Engineering Contradiction:

Improveviewing perspective freedomVSAvoiduser experience

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent merges the video playback function with the augmented reality display function into a single integrated system. The foreground object from the video is extracted and displayed as an AR object, while the background is captured by a background capturing device and combined with the foreground object, eliminating the need for separate interactive players and slider bars, thus improving user experience while maintaining viewing perspective freedom

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent transitions from traditional two-dimensional video playback to three-dimensional augmented reality display. By converting pixel points from the image coordinate system to the augmented reality coordinate system and combining with background images captured from different perspectives, the system enables six-degree-of-freedom viewing, significantly enhancing viewing perspective freedom

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Adaptability or versatility

If pixel points are converted from image coordinate system to augmented reality coordinate system, then the viewing perspective freedom is enhanced, but the processing complexity increases

Engineering Contradiction:

Improveviewing perspective freedomVSAvoidcoordinate conversion processing

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent performs preliminary coordinate conversion by pre-calculating the transformation matrix between the image coordinate system and the augmented reality coordinate system. This transformation matrix is computed in advance based on the camera parameters and spatial relationship, allowing for efficient real-time conversion of pixel points without complex processing during video playback

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces a coordinate transformation matrix as an intermediary to bridge the image coordinate system and the augmented reality coordinate system. This matrix serves as a mediator that simplifies the complex relationship between different coordinate systems, enabling straightforward pixel point conversion while reducing processing complexity

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If background image is combined with target image to create augmented reality image, then the stereoscopic impact is improved, but the image processing time increases

Engineering Contradiction:

Improvestereoscopic impactVSAvoidimage processing time

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent segments the image processing into distinct independent steps: foreground object extraction, coordinate conversion, background image capture, and final combination. By dividing the complex image processing into separate modular operations, each step can be optimized independently, reducing overall processing time while maintaining high stereoscopic impact

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements continuous image processing where the background capturing device operates simultaneously with the video playback, and the coordinate conversion is performed in real-time without interrupting the video stream. This continuous processing approach eliminates idle time and ensures smooth transitions between different viewing perspectives, reducing perceived processing time

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS20250061665A1Image display method, electronic device and storage medium

Publication Date: 2025.02.20 DOUYIN VISION CO LTD

AI summary

Embodiments of the present disclosure provided an image display method, apparatus, electronic device and storage medium. The method includes: acquiring a converted image corresponding to each video frame in a target video; acquiring a background pose of a background capturing device at a target moment, and determining a perspective image corresponding to the background pose from at least one converted image corresponding to the target moment; converting a pixel point in the perspective image into a background capturing coordinate system where the background capturing device is located according to the background pose to obtain a target image; combining a background image captured by the background capturing device at the target moment with the target image to obtain an augmented reality image, and displaying the augmented reality image.