Out-of-focus 3D camera

Abstract

The invention discloses an out-of-focus 3D camera, and the camera comprises a light splitting device which is used for dividing the incident light of a to-be-measured object into two light paths; the camera module is arranged on a transmission path of the light path, and after the light splitting path passes through a lens in the camera module, an out-of-focus image is formed on the light sensing device at the tail end; and the control device is used for setting the working mode of the system, obtaining the double-defocus image of the light sensing device in the camera module and the defocus distance of the distance measuring device in the camera module, and obtaining the depth information of the object to be measured through optimization calculation. According to the invention, the problems of high cost and the like of a real-time depth measurement device in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the field of depth measurement, and in particular to a defocused 3D camera. Background Art

[0002] With the development and maturity of digital image processing, precision optical equipment and image acquisition equipment, depth measurement technology based on image acquisition devices has developed rapidly. In the field of depth measurement technology, there are many existing methods, including: active depth measurement based on structured light fringe projection; focus operator measurement based on industrial cameras to obtain scene focus stack images; and two-dimensional image depth estimation based on deep learning.

[0003] However, the above-mentioned depth measurement technology is not suitable for real-time and convenient scene depth measurement due to the defects of the method itself. The specific defects are as follows: the active depth measurement method based on structured light fringe projection cannot be widely popularized because the experiment requires additional light sources and the measurement equipment is expensive; the focusing operator measurement method based on the scene focused stack image relies on a large number of image stacks and textures, and the effect is unstable; the two-dimensional image depth estimation method based on deep learning requires image preprocessing and a long training process, and most of them rely on real depth, making it difficult to obtain results quickly.

[0004] Currently, no effective solution has been proposed to address the problems that related depth measurement methods rely on multiple defocused shots, are time-consuming, do not combine the image acquisition process with the camera defocus depth optimization algorithm, cannot quickly obtain depth, and have high measurement costs. Summary of the invention

[0005] The present invention provides a defocused 3D camera to solve the problems in the related art of relying on multiple defocused shooting, which is time-consuming, not combining the image acquisition process with the camera defocus depth optimization algorithm, and being unable to quickly obtain the depth, and having a high measurement cost in the related method.

[0006] According to one aspect of the present invention, a defocused 3D camera is provided, comprising: a spectroscopic device, a camera module, and a control device, wherein the spectroscopic device is used to split the incident light of the object to be measured into two groups of light paths; the camera module is used to receive the light path projected by the object to be measured and perform defocused image acquisition, and the camera module includes at least two groups of acquisition devices; the control device is used to determine the working mode of the defocused 3D camera, obtain the defocused image and defocus distance of the object to be measured, and obtain the depth measurement information of the scene through a depth optimization algorithm.

[0007] According to another aspect of an embodiment of the present invention, a defocused 3D camera depth measurement method is provided, comprising: using the spectroscopic device and the camera module to collect dual defocus images and corresponding defocus distances in the same scene; a control device performs a blind deconvolution algorithm on the collected defocus images to obtain a clear image of the corresponding scene, and the clear image is combined with a given initial scene depth and a defocus depth obtained by a ranging device to generate a reconstructed defocus image through a convolution synthesis algorithm; calculating a difference loss value between the reconstructed defocus image and a real defocus image obtained by the camera under corresponding parameters, optimizing the initial scene depth according to the loss value, and obtaining an accurate scene depth after iterative calculation.

[0008] Optionally, when the object to be measured is in any mode, at least two camera modules of the defocused 3D camera work.

[0009] Optionally, at least two sets of image information are synchronously collected two-dimensional image data, and depth measurement values ​​of the object to be measured are obtained based on the defocus information of at least two sets of camera modules, including: performing image acquisition on the two-dimensional data of at least two sets of defocused images to obtain defocus distances of at least two sets of defocused images; and obtaining the depth of the object to be measured through an optimization algorithm based on at least two sets of defocus images and defocus distances.

[0010] In the examples of the present invention, the spectroscopic device in the defocused 3D camera splits the incident light into two, and projects the light into two groups of light paths into the camera module; through at least two groups of camera modules, a two-dimensional defocus image and a defocus distance of the object to be measured are acquired in the same scene; the control device performs algorithmic processing on the defocus image and the defocus distance, and obtains the depth information of the object to be measured by optimizing the calculation, thereby achieving the purpose of completing image acquisition and three-dimensional depth measurement in a short time, and achieving the technical effect of quickly and accurately obtaining the depth value of the object to be measured by relying on a few images, thereby solving the problems of relying on multiple defocused shooting, long time consumption, and high measurement cost in related depth measurement methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 is a workflow diagram of a defocused 3D camera provided according to an embodiment of the present application;

[0013] Figure 2 is a schematic diagram of a calculation flow of a defocused 3D camera depth measurement method provided in an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of the operation flow of the defocused 3D camera depth measurement method provided in an embodiment of the present application;

[0015] The above drawings include the following reference numerals:

[0016] 1. Object to be measured; 2. Spectroscopic device; 3a. Lens group 1; 3b. Lens group 2; 4a. Distance measuring device 1; 4b. Distance measuring device 2; 5a. Photosensitive device 1; 5b. Photosensitive device 2; 6. Control device. DETAILED DESCRIPTION

[0017] The present invention is further described below in conjunction with the embodiments, but it should not be understood that the above subject matter of the present invention is limited to the following embodiments. Without departing from the above technical ideas of the present invention, various substitutions and changes are made according to the common technical knowledge and customary means in the art, which should all be included in the protection scope of the present invention.

[0018] All other embodiments obtained by technicians without making any creative work should fall within the scope of protection of this application.

[0019] According to an embodiment of the present application, a defocused 3D camera is provided.

[0020] Figure 1 is a flowchart of a defocused 3D camera according to an embodiment of the present application. Figure 1 As shown, the defocused 3D camera includes: a spectroscopic device, a camera module and a control device.

[0021] A light splitting device, used to split the incident light of the object to be measured into two groups of light paths;

[0022] A camera module, used for receiving the light path projected by the object to be measured and collecting defocused images, and the camera module includes at least two groups of equipment;

[0023] The control device is used to determine the working mode of the defocused 3D camera, obtain the defocused image of the object to be measured from the photosensitive device in the camera module and obtain the defocused distance from the distance measuring device, take the double defocused image and the defocused distance as input, and obtain the depth measurement information of the object to be measured through the depth optimization algorithm.

[0024] Optionally, in the defocused 3D camera provided in the embodiment of the present application, the light splitting device can be any one of the following: one or more flat beam splitters, one or more cubic beam splitters. The light splitting device can split the incident light of the scene into two or more groups of light paths according to a given angle and a specified ratio.

[0025] Alternatively, if Figure 2As shown, in the defocused 3D camera provided in the embodiment of the present application, when there is only one light splitting device, the camera module is arranged on the transmission path of the light splitting path emitted by the light splitting device, and the camera module includes a lens group, a distance measuring device and a photosensitive device. The lens group is located in the two light paths, and is used to converge the two groups of light separated by the light splitter onto the two photosensitive imaging devices; the distance measuring device has two sets of distance measuring devices, which are used to measure the distance from the optical center of the lens group to the photosensitive imaging device; and the photosensitive device has two sets of photosensitive imaging elements, which are used for photosensitive imaging.

[0026] The embodiment of the present application also provides a defocused 3D camera depth measurement method. It should be noted that the defocused 3D camera of the embodiment of the present application can be used to perform the defocused 3D camera depth measurement method provided by the embodiment of the present application. The defocused 3D camera depth measurement method provided by the embodiment of the present application is introduced below.

[0027] Figure 3 is a schematic diagram of the operation flow of the defocused 3D camera depth measurement method according to an embodiment of the present application, such as Figure 3 As shown, the method comprises the following steps:

[0028] Step 1: arrange the relative positions of the spectrometer and the camera module, control the light flux of the two light paths divided by the spectrometer to be consistent, and collect a double defocus image of the same scene through the spectrometer and the camera module.

[0029] Step 2: The control device performs a blind deconvolution operation on the defocus image collected by the camera module's photosensitive device to obtain a scene clarity image, and performs a convolution synthesis operation on the scene clarity image with the defocus distance and initial depth obtained by the ranging device to obtain a reconstructed defocus image.

[0030] Step 3: Perform initial depth optimization calculation based on the difference loss between the reconstructed defocus image and the defocus image collected by the camera module to obtain the depth value of the object to be measured.

[0031] Optionally, in the defocused 3D camera depth measurement method provided in an embodiment of the present application, the blind deconvolution operation method includes but is not limited to any one of the following: an inverse filtering algorithm and a regularization algorithm.

[0032] Optionally, in the defocused 3D camera depth measurement method provided in an embodiment of the present application, the convolution synthesis operation method includes but is not limited to a convolution operation based on a point spread function.

[0033] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A defocused 3D camera, characterized in that: The measuring device comprises: A spectrometer is used to split the incident light of the object to be measured into two groups to obtain two groups of light paths with the same light output; The camera module is arranged on the transmission path of the optical path, wherein after the incident light passes through the camera module, the required image information of the scene to be measured will be generated. The control device is used to determine the working mode of the defocused 3D camera, obtain the light-sensitive image and defocus distance of the object to be measured, and obtain the depth information of the object to be measured by calculation. The camera module includes a lens group, a distance measuring device and a photosensitive device. The lens group is located in the optical path and is used to converge the light separated by the light splitting device onto the photosensitive device; the distance measuring device is used to measure the defocus distance from the optical center of the lens to the photosensitive imaging device; and the photosensitive device has a photosensitive imaging element, which is located at the end of the camera device and is used for photosensitive imaging.

2. The defocused 3D camera according to claim 1, characterized in that: The light splitting device further comprises any one of the following: One or more flat beam splitters, dividing the incident light into two or more groups of required light paths according to a given angle and a specified ratio; One or more cubic beam splitters divide the incident light into two or more groups of required light paths according to a given angle and a specified ratio.

3. The camera module according to claim 1, characterized in that: The photosensitive plate type of the photosensitive device is CCD or CMOS.

4. The camera module according to claim 1, characterized in that: The distance measuring device also includes any one of the following: A laser distance measuring device is used to obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the defocus depth during image acquisition; The infrared distance measuring device obtains the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the defocus depth during image acquisition.

5. The defocused 3D camera according to claim 1, characterized in that: The control device is specifically used for: Controlling the working state of the camera module; Controlling the camera module to form an image, and obtaining a defocus image obtained by the photosensitive device and a defocus distance obtained by the distance measuring device; The acquired defocus map and defocus distance are input into a depth solving algorithm to obtain a depth measurement result.

6. A defocused 3D camera image acquisition method, characterized in that: The collection method comprises: The incident light of the object to be measured is split by a spectroscopic device and projected to the camera module in the form of two or more light paths. At least two groups of camera modules obtain image information of the object to be measured at different defocus distances. The image information is at least two groups of image information. Wherein, the defocused 3D camera image acquisition method is executed by the defocused 3D camera described in claim 1.

7. A method for measuring depth of a defocused 3D camera, characterized in that: The depth measurement method comprises: The incident light of the object to be measured is projected to the camera module through the light splitting device, and the control device controls the photosensitive device of the camera module to form an image, and collects the defocus image obtained by the photosensitive device and the defocus distance obtained by the distance measuring device; The defocus image is processed by a blind deconvolution algorithm to obtain a clear image of the corresponding scene, and the clear image is combined with a given initial scene depth and the defocus depth obtained by the ranging device to generate a corresponding reconstructed defocus image through a convolution synthesis algorithm; The reconstructed defocus image is compared with the real defocus image obtained by the camera under corresponding parameters, the difference loss value between the images is calculated, and the initial scene depth is updated and optimized according to the loss value, and an accurate scene depth value is obtained after optimization. Wherein, the defocused 3D camera depth measurement method is performed by the defocused 3D camera described in claim 1.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the method according to any one of claims 6 and 7.

9. A processor, characterized in that: The processor is used to run a program, and the program executes the method according to any one of claims 6 and 7.

Images