Image Processing Server Model Data Segmentation for Virtual Viewpoint Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for distributing three-dimensional image data for virtual viewpoint images face challenges such as large data amounts leading to increased latency and bandwidth occupation, image quality degradation due to compression, and increased processing loads, especially in wireless communication environments.
Innovation Solution
The proposed solution involves an image processing server that obtains model data and viewpoint information, generates and transmits image data by rendering a portion of the model data based on received viewpoint information, while categorizing model data for efficient distribution using both model streaming and image streaming techniques, optimizing bandwidth usage and image quality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If model data is distributed to multiple user terminals using wireless communication, then interactive virtual viewpoint image data can be generated with reduced delay, but the large data amount occupies bandwidth and increases transmission delay
Solution Approach 1:
The model data is divided into multiple portions, with a first portion transmitted to user terminals for rendering while a second portion remains on the server. This segmentation allows interactive rendering at terminals without requiring complete model data transmission, resolving the bandwidth occupation issue while maintaining interactivity.
Solution Approach 2:
The patent transitions from a single-dimensional approach (complete model data transmission) to a multi-dimensional distribution strategy where different portions of model data are allocated to different locations (server and terminals). This enables flexible bandwidth management and reduced transmission delays.
2Quantity of substance
If the image processing server performs rendering and transmits compressed image data, then communication data amount is reduced, but image quality degrades due to compression noise
Solution Approach 1:
Model data is segmented into first portion (transmitted for rendering) and second portion (retained on server). This allows terminals to perform local rendering with higher quality than compressed images while still reducing communication data compared to transmitting complete model data.
Solution Approach 2:
The patent changes the parameter of data representation from compressed image format to raw model data portion, enabling lossless or higher quality reconstruction at the terminal while maintaining reduced communication bandwidth usage.
3Quantity of substance
If the image processing server obtains control information from each user terminal to perform rendering, then communication data amount is reduced, but processing load of the server increases
Solution Approach 1:
Rendering tasks are segmented between server and terminals. The server handles only the second portion of model data and control information processing, while terminals perform the actual rendering of the first portion. This distributes processing load and reduces server complexity.
Solution Approach 2:
User terminals perform rendering operations themselves using the first portion of model data, making them self-sufficient for the rendering task. This eliminates the need for the server to perform complex rendering operations for each terminal.
Data Source
AI summary
A technique of the present disclosure includes: an obtaining unit configured to obtain model data indicating a three-dimensional shape of an object in a space; a receiving unit configured to receive viewpoint information from an external apparatus, the viewpoint information indicating a virtual viewpoint for a virtual viewpoint image to be generated by the external apparatus based on data transmitted from the image processing apparatus, the virtual viewpoint image depicting the space as viewed from the virtual viewpoint; a generating unit configured to generate image data by rendering a first portion of the obtained model data, based on the received viewpoint information; and a transmitting unit configured to transmit the generated image data and a second portion of the obtained model data different from the first portion to the external apparatus.