AR Streaming Device Edge Server Segmentation Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) services face challenges in providing real-time streaming due to high data processing requirements and inability to support lightweight devices with ultra-low delay communication.
Innovation Solution
An AR streaming device and system that interoperate with an edge server, performing segmentation rendering and video transmission using QUIC protocol, multiplexing and de-multiplexing based on ISO 23000-19 CMAF standard, and decoding and blending AR video for real-time streaming.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If AR services use traditional video transmission methods, then video quality can be maintained, but real-time streaming is not achieved due to high data processing requirements and communication delay
Solution Approach 1:
The patent segments the video transmission process into discrete media chunks using CMAF format, allowing selective transmission and processing. The edge server receives image data, performs segmentation rendering, and transmits only the necessary AR video segments back to the device, reducing unnecessary data transmission and enabling real-time streaming.
Solution Approach 2:
The patent introduces an edge server as an intermediary between the user device and the content source. The edge server receives image data from the device, performs segmentation rendering, and transmits the processed AR video back to the device. This intermediary approach enables real-time processing and reduces communication delay by pre-processing and optimizing the video stream.
2Productivity
If AR services process all video data locally, then real-time rendering is achieved, but device resources are overwhelmed and lightweight devices cannot support the service
Solution Approach 1:
The patent extracts the computationally intensive segmentation rendering task from the user device and relocates it to the edge server. The user device only transmits image data and receives processed AR video, significantly reducing the processing burden on lightweight devices while maintaining real-time rendering capability through the edge server's computational power.
Solution Approach 2:
The patent uses a simplified video reproduction approach where the edge server generates AR video content that is then copied and displayed on the user device. Instead of requiring the device to perform complex rendering operations, the system transmits pre-processed video segments that can be directly displayed, reducing resource consumption while maintaining visual quality.
3Manufacturing precision
If AR services use complex video processing methods, then video quality is maintained, but device complexity increases and implementation becomes difficult
Solution Approach 1:
The patent changes the parameter format from traditional continuous video streams to discrete media chunks following CMAF standards. This parameter change simplifies the transmission and processing requirements, allowing standard video processing techniques to be applied to AR content while maintaining quality and reducing implementation complexity.
Solution Approach 2:
Instead of having the user device perform complex rendering and then transmit the video, the patent inverts the approach by having the edge server receive image data, perform the complex segmentation rendering, and transmit the processed AR video back to the device. This inversion simplifies the device-side implementation while maintaining video quality through the edge server's processing capabilities.
Data Source
AI summary
An augmented reality (AR) streaming device interoperating with an edge server is proposed. The AR streaming device may include a sensing module including a camera and a certain inertia sensor, a display module streaming AR video, a communication module transmitting or receiving data to or from the edge server and the sensing module. The device may also include a memory and a processor executing a program stored in the memory. When image data and inertia data are obtained through the sensing module, the processor may perform synchronization and encoding on the image data and the inertia data to transmit to the edge server through the communication module. When segmentation rendering-processed AR video is received from the edge server, the processor may perform decoding and blending on the segmentation rendering AR video to perform control to be streamed through the display module.