Image Processing Apparatus Reducing Data Transmission for Virtual Viewpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of virtual viewpoint images using multiple cameras results in a significant amount of data transmission, which burdens communication paths and server processing loads, particularly when generating virtual viewpoint contents that include both images and audio.
Innovation Solution
An image processing apparatus that employs adapters to separate foreground images from captured images, using difference images between camera views, thereby reducing the data amount transmitted between devices, with adapters connected in a daisy-chain configuration to efficiently process and transmit only the necessary foreground data to a server.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If all captured images from multiple cameras are transmitted to the server for virtual viewpoint generation, then the completeness of image data is improved, but the data transmission amount and server processing load increase significantly
Solution Approach 1:
The patent extracts only the essential foreground information from captured images by generating difference images that represent changes between consecutive frames. Instead of transmitting complete captured images, the system transmits only the difference images containing foreground object information, thereby reducing data transmission amount while maintaining the necessary information for virtual viewpoint generation
Solution Approach 2:
The patent segments the image data transmission process by dividing it into two parts: difference images transmitted from image capturing apparatus to image processing apparatus, and foreground images generated and transmitted from image processing apparatus to server. This segmentation allows intermediate processing at the image processing apparatus to reduce the overall data burden on the server
2Measurement precision
If the server receives and processes all captured images directly, then the processing accuracy is improved, but the server processing load increases significantly
Solution Approach 1:
The patent performs preliminary processing of captured images at the image processing apparatus by generating difference images and extracting foreground information before transmitting to the server. This preliminary action reduces the complexity of data received by the server, thereby lowering the server's processing load while maintaining the accuracy needed for virtual viewpoint generation
Solution Approach 2:
The image processing apparatus acts as an intermediary between the image capturing apparatus and the server. It receives captured images, processes them to generate difference images and foreground images, and then transmits the processed data to the server. This intermediary role distributes the processing burden and protects the server from handling raw high-volume captured image data
3Quantity of substance
If difference images are used instead of complete images for transmission, then the data transmission amount is reduced, but the complexity of image processing increases
Solution Approach 1:
The image processing apparatus performs self-service by automatically generating difference images from captured images and subsequently generating foreground images from those difference images. This self-service capability at the intermediate device eliminates the need for manual processing and distributes the computational complexity away from the server, making the overall system more efficient
Data Source
AI summary
An image processing apparatus in an image processing system configured to generate a virtual point image using a plurality of images captured by a plurality of image capturing apparatus receives a first difference image being based on a difference between a third image based on image capturing performed by a third image capturing apparatus of the plurality of image capturing apparatuses and a second image based on image capturing performed by a second image capturing apparatus of the plurality of image capturing apparatuses and has a smaller data amount than that of the third image, and acquires a foreground image by extracting a predetermined object from the second image, using the received first difference image and a second difference image between a first image based on image capturing performed by a first image capturing apparatus of the plurality of image capturing apparatus and the second image.