A 3D camera-based information acquisition device and method

Abstract

The present invention discloses an information acquisition device and method based on a 3D camera, including multiple 3D cameras, a video acquisition processor and an image display; the multiple 3D cameras are used for real-time video recording and shooting, and the real-time video is converted into digital information and stored in a data buffer; the video acquisition processor is internally encapsulated with a data acquisition module, a data conversion module, a data processing module and a model building module; the data acquisition module reads the digital signal in the 3D camera buffer in real time, and what is read is the current frame. When acquiring 3D camera information, the present invention performs secondary processing compensation on the acquired 3D camera information data, and simultaneously obtains the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera as modeling auxiliary information, which effectively ensures the true restoration degree of the acquired 3D camera information in the 3D modeling process, and ensures the clarity of the image information acquired by the 3D camera.

Description

Technical Field

[0001] The present invention relates to an information acquisition method of a 3D camera, and in particular to an information acquisition device and method based on a 3D camera, belonging to the technical field of 3D cameras. Background Art

[0002] 3D cameras are cameras made with 3D lenses, usually with two or more camera lenses, with a spacing close to that of human eyes, and can capture different images of the same scene similar to what the human eye sees. Holographic 3D has more than 5 lenses, and can view the same image in all directions through dot grating imaging or grating holographic imaging, making it feel like being there in person. The function of the 3D camera itself is like the human brain, which can fuse two images together into a 3D image.

[0003] Currently, when acquiring information from a 3D camera, the accuracy of data information is easily lost when the video signal is converted into a digital signal. At the same time, no reference is obtained for the external parameters of the 3D camera. When constructing a 3D model using the acquired 3D camera signal, the video shot by the 3D camera cannot be truly restored, and a deviation occurs between the constructed 3D model and the shot. Summary of the invention

[0004] The object of the present invention is to provide an information acquisition device and method based on a 3D camera to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solution: comprising a plurality of 3D cameras, a video acquisition processor and an image display;

[0006] The plurality of 3D cameras are used for real-time video recording and converting the real-time video into digital information and storing it in a data buffer;

[0007] The video acquisition processor internally encapsulates a data acquisition module, a data conversion module, a data processing module and a model building module;

[0008] Data acquisition module: reads the digital signal in the 3D camera buffer in real time, and reads the current frame;

[0009] Data processing module: converts the read digital signal into 2D image information, extracts the texture features of the 2D image information, and sends the feature parameters to the model building module and camera data for secondary processing;

[0010] Data conversion module: converts the read digital signal into a video signal and sends the converted video signal to the model building module;

[0011] Model building module: The model building module receives characteristic parameters of the video signal and the video image, and builds a 3D model of the current picture according to the characteristic parameters of the video signal and the video image;

[0012] The image display: the video acquisition processor sends the constructed current 3D model to the image display, and the image display displays the 3D image.

[0013] As a preferred technical solution of the present invention, the data acquisition module also includes the inclination angle of the 3D camera, the direction and data size of the 3D camera, and the focal length of the 3D camera;

[0014] Obtaining the inclination angle of the 3D camera: using Zhang's calibration method to obtain the inclination angle of the current 3D camera;

[0015] The direction and data size of the 3D camera: set the direction of the camera data preview and the current camera data size, and obtain the direction coordinates and data size of the current camera;

[0016] The focal length of the 3D camera: obtain the focal length of the current 3D camera through autoFocus;

[0017] As a preferred technical solution of the present invention, the camera data is processed again: the Tenengrad gradient method is used to use the Sobel operator to calculate the gradients of the image in the horizontal and vertical directions respectively, and the gradients in the horizontal and vertical directions are compensated.

[0018] As a preferred technical solution of the present invention, the texture features of the image are extracted using grayscale difference statistics method. The specific method is as follows:

[0019] Let g(x,y) be the pixel value at position (x,y) in the 2D image, Δx and Δy be the offsets in the x and y directions respectively, then the grayscale difference between (x,y) and its slightly offset point (x+Δx,y+Δy) is g Δ (x,y)=g(x,y)-g(x+Δx,y+Δy).

[0020] As a preferred technical solution of the present invention, the model construction includes the following steps:

[0021] Step 1: Calculate the features of each pixel of the 2D image;

[0022] Step 2: Match the pixels of multiple images to obtain the features of the images and the parameters of the 3D camera to obtain sparse 3D information;

[0023] Step 3: Based on the camera parameters obtained in the second step, perform dense reconstruction to obtain a dense point cloud;

[0024] Step 4: Perform coordinate construction processing on the point cloud in the third step to obtain a grid, and then construct the grid.

[0025] As a preferred technical solution of the present invention, the feature of each pixel of the 2D image is calculated using the SFM method, and the dense point cloud is calculated using the CMVS / PMVS method.

[0026] A method for obtaining information based on a 3D camera comprises the following steps:

[0027] S1: Connect the acquisition device, connect multiple 3D cameras to the video acquisition processor signal, and connect the video acquisition processor to the image display signal;

[0028] S2: The 3D camera records video in real time and converts the recorded video into digital signals and stores them in the data buffer. The data acquisition module reads the digital signals in the 3D camera buffer in real time and simultaneously acquires the inclination angle, direction and data size of the 3D camera and the focal length of the 3D camera.

[0029] S3: The data processing module provided in the video acquisition processor converts the read digital signal into 2D image information, extracts the texture features of the 2D image information image and performs secondary processing on the camera data, converts the read digital signal into a video signal, and sends the converted video signal to the model building module;

[0030] S4: through the model building module, the secondary processing data of the camera data, the image texture features, the video information, the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera are obtained to build the model;

[0031] S5: Send the constructed model to an image display to display a 3D image.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention discloses an information acquisition device and method based on a 3D camera. When acquiring 3D camera information, secondary processing compensation is performed on the acquired 3D camera information data, and the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera are simultaneously obtained as modeling auxiliary information, thereby effectively ensuring the true restoration degree of the acquired 3D camera information in the 3D modeling process and ensuring the clarity of the image information acquired by the 3D camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural schematic diagram of the present invention;

[0035] Figure 2 It is a flowchart of the present invention; DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] See also Figure 1-2 The present invention provides a technical solution of an information acquisition device and method based on a 3D camera:

[0038] according to Figure 1-2 As shown, it includes multiple 3D cameras, a video acquisition processor and an image display;

[0039] The plurality of 3D cameras are used for real-time video recording and converting the real-time video into digital information and storing it in a data buffer;

[0040] The video acquisition processor internally encapsulates a data acquisition module, a data conversion module, a data processing module and a model building module;

[0041] Data acquisition module: reads the digital signal in the 3D camera buffer in real time, and reads the current frame;

[0042] Data processing module: converts the read digital signal into 2D image information, extracts the texture features of the 2D image information, and sends the feature parameters to the model building module and camera data for secondary processing;

[0043] Data conversion module: converts the read digital signal into a video signal and sends the converted video signal to the model building module;

[0044] Model building module: The model building module receives characteristic parameters of the video signal and the video image, and builds a 3D model of the current picture according to the characteristic parameters of the video signal and the video image;

[0045] Image display: The video acquisition processor sends the constructed current 3D model to the image display, and the image display displays the 3D image.

[0046] The data acquisition module also includes the inclination angle of the 3D camera, the direction and data size of the 3D camera, and the focal length of the 3D camera;

[0047] Obtaining the inclination angle of the 3D camera: using Zhang's calibration method to obtain the inclination angle of the current 3D camera;

[0048] 3D camera direction and data size: set the direction of camera data preview and the current camera data size, and obtain the direction coordinates and data size of the current camera;

[0049] Focal length of 3D camera: Get the focal length of the current 3D camera through autoFocus;

[0050] Secondary processing of camera data: The Tenengrad gradient method is used to calculate the horizontal and vertical gradients of the image using the Sobel operator, and the horizontal and vertical gradients are compensated.

[0051] The texture features of the image are extracted using the grayscale difference statistics method. The specific method is as follows:

[0052] Let g(x,y) be the pixel value at position (x,y) in the 2D image, Δx and Δy be the offsets in the x and y directions respectively, then the grayscale difference between (x,y) and its slightly offset point (x+Δx,y+Δy) is g Δ (x,y)=g(x,y)-g(x+Δx,y+Δy).

[0053] Model building includes the following steps:

[0054] Step 1: Calculate the features of each pixel of the 2D image. The SFM method is used to calculate the features of each pixel of the 2D image.

[0055] Step 2: Match the pixels of multiple images to obtain the features of the images and the parameters of the 3D camera to obtain sparse 3D information;

[0056] Step 3: Based on the camera parameters obtained in the second step, perform dense reconstruction to obtain a dense point cloud. The dense point cloud uses the CMVS / PMVS method.

[0057] Step 4: Perform coordinate construction processing on the point cloud in the third step to obtain a grid, and then construct the grid.

[0058] By collecting the texture features of the image, it is easier to truly restore the image features when building a 3D model.

[0059] An information acquisition device based on a 3D camera comprises the following steps:

[0060] S1: Connect the acquisition device, connect multiple 3D cameras to the video acquisition processor signal, and connect the video acquisition processor to the image display signal;

[0061] S2: The 3D camera records video in real time and converts the recorded video into digital signals and stores them in the data buffer. The data acquisition module reads the digital signals in the 3D camera buffer in real time and simultaneously acquires the inclination angle, direction and data size of the 3D camera and the focal length of the 3D camera.

[0062] S3: The data processing module provided in the video acquisition processor converts the read digital signal into 2D image information, extracts the texture features of the 2D image information image and performs secondary processing on the camera data, converts the read digital signal into a video signal, and sends the converted video signal to the model building module;

[0063] S4: through the model building module, the secondary processing data of the camera data, the image texture features, the video information, the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera are obtained to build the model;

[0064] S5: Send the constructed model to an image display to display a 3D image.

[0065] When acquiring 3D camera information, secondary processing compensation is performed on the acquired 3D camera information data, and the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera are obtained as auxiliary modeling information at the same time, which effectively ensures the true restoration degree of the acquired 3D camera information in the 3D modeling process and ensures the clarity of the image information acquired by the 3D camera.

[0066] In the description of the present invention, it is necessary to understand that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the accompanying drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0067] In the present invention, unless otherwise clearly stipulated and limited, for example, it can be a fixed connection, a detachable connection, or an integrated one; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. Unless otherwise clearly defined, for ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0068] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An information acquisition device based on a 3D camera, characterized in that: Includes multiple 3D cameras, video acquisition processors and image displays; The plurality of 3D cameras are used for real-time video recording and converting the real-time video into digital information and storing it in a data buffer; The video acquisition processor internally encapsulates a data acquisition module, a data conversion module, a data processing module and a model building module; Data acquisition module: reads the digital signal in the 3D camera buffer in real time, and reads the current frame; Data processing module: converts the read digital signal into 2D image information, extracts the texture features of the 2D image information, and sends the feature parameters to the model building module and camera data for secondary processing; Data conversion module: converts the read digital signal into a video signal and sends the converted video signal to the model building module; Model building module: The model building module receives characteristic parameters of the video signal and the video image, and builds a 3D model of the current picture according to the characteristic parameters of the video signal and the video image; The image display: the video acquisition processor sends the constructed current 3D model to the image display, and the image display displays the 3D image; The camera data is processed twice: the gradients of the image in the horizontal and vertical directions are calculated respectively by using the Tenengrad gradient method and the Sobel operator, and the gradients in the horizontal and vertical directions are compensated.

2. The information acquisition device based on a 3D camera according to claim 1, characterized in that: The data acquisition module also includes the inclination angle of the 3D camera, the direction and data size of the 3D camera, and the focal length of the 3D camera; Obtaining the inclination angle of the 3D camera: using Zhang's calibration method to obtain the inclination angle of the current 3D camera; The direction and data size of the 3D camera: set the direction of the camera data preview and the current camera data size, and obtain the direction coordinates and data size of the current camera; The focal length of the 3D camera: obtain the focal length of the current 3D camera through autoFocus.

3. The information acquisition device based on a 3D camera according to claim 2, characterized in that: The texture features of the image are extracted using the grayscale difference statistics method. The specific method is as follows: Let g(x,y) be the pixel value at position (x,y) in the 2D image, Δx and Δy be the offsets in the x and y directions respectively, then the grayscale difference between (x,y) and its slightly offset point (x+Δx,y+Δy) is g Δ (x,y)=g(x,y)-g(x+Δx,y+Δy).

4. The information acquisition device based on a 3D camera according to claim 3, characterized in that: The model construction includes the following steps: Step 1: Calculate the features of each pixel of the 2D image; Step 2: Match the pixels of multiple images to obtain the features of the images and the parameters of the 3D camera to obtain sparse 3D information; Step 3: Based on the camera parameters obtained in the second step, perform dense reconstruction to obtain a dense point cloud; Step 4: Perform coordinate construction processing on the point cloud in the third step to obtain a grid, and then construct the grid.

5. The information acquisition device based on a 3D camera according to claim 4, characterized in that: The feature of each pixel of the 2D image is calculated by using the SFM method, and the dense point cloud is calculated by using the CMVS / PMVS method.

6. A method for obtaining information based on a 3D camera, characterized in that: An information acquisition device based on a 3D camera according to any one of claims 1 to 5, comprising the following steps: S1: Connect the acquisition device, connect multiple 3D cameras to the video acquisition processor signal, and connect the video acquisition processor to the image display signal; S2: The 3D camera records video in real time and converts the recorded video into digital signals and stores them in the data buffer. The data acquisition module reads the digital signals in the 3D camera buffer in real time and simultaneously acquires the inclination angle, direction and data size of the 3D camera and the focal length of the 3D camera. S3: The data processing module provided in the video acquisition processor converts the read digital signal into 2D image information, extracts the texture features of the 2D image information image and performs secondary processing on the camera data, converts the read digital signal into a video signal, and sends the converted video signal to the model building module; S4: through the model building module, the secondary processing data of the camera data, the image texture features, the video information, the inclination angle of the 3D camera, the direction and data size of the 3D camera and the focal length information of the 3D camera are obtained to build the model; S5: Send the constructed model to an image display to display a 3D image.

Images