One billion-pixel virtual reality video collection device, system and method

Abstract

The invention provides a billion-pixel virtual reality video collection device, system and method. The device comprises an unstructured camera array, a supporting part, a camera holder and a camera rack, wherein the camera array comprises at least five columns of camera column combinations which are distributed in a fan shape, each camera column combination comprises a binocular camera and at least one local camera, each binocular camera is composed of two global cameras as a group, and the focal length of each local camera is adjustable; the supporting part is used for supporting the camera holder, and the camera holder is connected with the camera rack; and the camera rack is used for respectively fixing each column of camera column combination in the camera array through a connecting part. A panoramic 360 virtual reality collection device with a self-adaptive unstructured structure is used; and embedding an RGB image in the video data of the local camera into the panorama, so that auser can zoom in a visual angle to observe detail information of the area of interest.

Description

technical field [0001] The present invention relates to the technical field of gigapixel virtual reality video acquisition, in particular to a gigapixel virtual reality video acquisition device, system and method. Background technique [0002] Among the five major perception pathways of human beings, vision accounts for 70% to 80% of the information sources; and about 50% of the brain's ability is used for visual information and perception. Existing image acquisition and display loses multiple dimensions of visual information. This forces us to use only two-dimensional "windows" to observe the three-dimensional world. The human eye can see the three-dimensional world because the human eye receives the light emitted by the object (active or passive light emission), and the light field is a complete representation of the light collection in the three-dimensional world. The light field display can completely reproduce the real world visually, but the light field must first be...

AI Technical Summary

Problems solved by technology

[0003] Traditional image acquisition based on camera arrays has the following disadvantages: (1) To ensure that the resolution is not lost, a large number of cameras are required, and the hardware cost is high; (2) Camera synchronization control is complicated, the amount of data is large, and the cost of storage and transmission is high ; (3) The traditional camera array requires sensors to follow a uniform distribution, and the calibration of a large number of cameras is complex and time-consuming; (4) The structured camera array arrangement does not support scene scaling at different scales, nor can it flexibly select specific areas to enhance resolution

For example, if a 4K (4096*2160) image sensor is used to capture a light field of 4x4 viewpoints, the image resolution will be reduced to a quarter of the original resolution in both horizontal and vertical directions, that is, the image resolution of a single viewpoint is only 1024*540; (2) Limited by the aperture of the camera, the field of parallax (FOP) angle of the collected light field is small, that is, the parallax is small, and the viewpoint can only be changed within a small angle range

[0005] Light field acquisition based on coded masks has the following disadvantages: (1) The light transmittance of the mask cannot reach 100%, so the light signal intensity will be lost, resulting in low imaging signal-to-noise ratio; (2) The final light field image reconstructed It is not directly collected by the imaging sensor, but obtained by demodulation from the modulated image; in essence, it is based on the learned light field dictionary to "guess" the light field to be reconstructed, and the result is not very confident

[0006] Based on the inherent contradiction of wide field of view and high resolution when recording videos from daily behaviors to complex operating states, the existing technology lacks a robust and high-quality light field acquisition method for virtual reality content generation

Images