A Gigapixel Virtual Reality Video Acquisition Device, System and Method

Abstract

The present invention provides a gigapixel virtual reality video acquisition device, system and method. The device includes: an unstructured camera array, a support member, a camera platform, and a camera rack; the camera array includes at least 5 columns distributed in a fan shape A combination of camera columns, each of which includes a binocular camera composed of two global cameras and at least one local camera, the focal length of the local camera is adjustable; the support is used to support the The camera platform, the camera platform, is connected to the camera frame; the camera frame is used to respectively fix each column of the camera array combination in the camera array through a connecting piece. Using a structure-adaptive and unstructured panorama 360 virtual reality acquisition device; and embedding the RGB image in the video data of the local camera into the panorama, the user can zoom in and zoom in to observe the details of the region of interest.

Description

technical field [0001] The present invention relates to the technical field of gigapixel virtual reality video acquisition, in particular to a gigapixel virtual reality video acquisition device, system and method. Background technique [0002] Among the five major perception pathways of human beings, vision accounts for 70% to 80% of the information sources; and about 50% of the brain's ability is used for visual information and perception. Existing image acquisition and display loses multiple dimensions of visual information. This forces us to use only two-dimensional "windows" to observe the three-dimensional world. The human eye can see the three-dimensional world because the human eye receives the light emitted by the object (active or passive light emission), and the light field is a complete representation of the light collection in the three-dimensional world. The light field display can completely reproduce the real world visually, but the light field must first be...

AI Technical Summary

Problems solved by technology

[0003] Traditional image acquisition based on camera arrays has the following disadvantages: (1) To ensure that the resolution is not lost, a large number of cameras are required, and the hardware cost is high; (2) Camera synchronization control is complicated, the amount of data is large, and the cost of storage and transmission is high ; (3) The traditional camera array requires sensors to follow a uniform distribution, and the calibration of a large number of cameras is complex and time-consuming; (4) The structured camera array arrangement does not support scene scaling at different scales, nor can it flexibly select specific areas to enhance resolution

For example, if a 4K (4096*2160) image sensor is used to capture a light field of 4x4 viewpoints, the image resolution will be reduced to a quarter of the original resolution in both horizontal and vertical directions, that is, the image resolution of a single viewpoint is only 1024*540; (2) Limited by the aperture of the camera, the field of parallax (FOP) angle of the collected light field is small, that is, the parallax is small, and the viewpoint can only be changed within a small angle range

[0005] Light field acquisition based on coded masks has the following disadvantages: (1) The light transmittance of the mask cannot reach 100%, so the light signal intensity will be lost, resulting in low imaging signal-to-noise ratio; (2) The final light field image reconstructed It is not directly collected by the imaging sensor, but obtained by demodulation from the modulated image; in essence, it is based on the learned light field dictionary to "guess" the light field to be reconstructed, and the result is not very confident

[0006] Based on the inherent contradiction of wide field of view and high resolution when recording videos from daily behaviors to complex operating states, the existing technology lacks a robust and high-quality light field acquisition method for virtual reality content generation

Images