A defocused 3D camera

Abstract

This invention discloses a defocus 3D camera, comprising: a beam splitter for splitting incident light from the object under test into two beam paths; a camera module disposed on the transmission path of the beam paths, wherein the beams, after passing through a lens in the camera module, form a defocus image on a photosensitive device at the end; and a control device for setting the system's operating mode, obtaining the dual defocus image from the photosensitive device in the camera module and the defocus distance from the ranging device in the camera module, and obtaining the depth information of the object under test through optimized calculation. This invention solves the problem of high cost in related technologies for real-time depth measurement devices.

Description

Technical Field

[0001] This invention relates to the field of depth measurement, and more specifically, to a defocused 3D camera. Background Technology

[0002] With the development and maturation of digital image processing, precision optical equipment, and image acquisition equipment, depth measurement technology based on image acquisition devices has developed rapidly. In the field of depth measurement technology, there are many existing methods, including: active depth measurement based on structured light fringe projection; focusing operator measurement based on scene focused stacked images obtained from industrial cameras; and two-dimensional image depth estimation based on deep learning.

[0003] However, the aforementioned depth measurement techniques are not suitable for real-time and convenient scene depth measurement due to their inherent limitations. Specifically, these limitations are as follows: Active depth measurement methods based on structured light fringe projection require additional light sources for experiments and have expensive measurement equipment, making them unsuitable for widespread adoption; Focusing operator measurement methods based on scene-focused stacked images rely on a large number of image stacks and textures, resulting in unstable performance; and Two-dimensional image depth estimation methods based on deep learning require image preprocessing and a long training process, and mostly rely on the actual depth, making it difficult to obtain results quickly.

[0004] There is currently no effective solution to the problems that related depth measurement methods rely on multiple out-of-focus shots, are time-consuming, do not combine the image acquisition process with the camera's defocus depth optimization algorithm, cannot quickly obtain depth, and have high measurement costs. Summary of the Invention

[0005] This invention provides an off-focus 3D camera to solve the problems of long time consumption due to multiple off-focus shooting in related technologies, lack of integration of image acquisition process with camera off-focus depth optimization algorithm, inability to quickly obtain depth, and high measurement cost in related methods.

[0006] According to one aspect of the present invention, an off-focus 3D camera is provided, comprising: a beam splitting device, a camera module, and a control device, wherein the beam splitting device is used to split the incident light rays of the object under test into two sets of light paths; the camera module is used to receive the light paths projected by the object under test and to perform off-focus image acquisition, and the camera module includes at least two sets of acquisition devices; the control device is used to determine the working mode of the off-focus 3D camera, obtain the off-focus image and off-focus distance of the object under test, and obtain the depth measurement information of the scene through a depth optimization algorithm.

[0007] According to another aspect of the present invention, a method for measuring the depth of a defocused 3D camera is provided, comprising: acquiring dual defocus images and corresponding defocus distances in the same scene using the beam splitting device and the camera module; processing the acquired defocus images using a blind deconvolution algorithm to obtain a corresponding scene sharp image using a control device; generating a reconstructed defocus image by combining the sharp image with a given initial scene depth and the defocus depth obtained by the ranging device through a convolution synthesis algorithm; calculating the difference loss value between the reconstructed defocus image and the real defocus image obtained by the camera under corresponding parameters; optimizing the initial scene depth based on the loss value; and obtaining an accurate scene depth after iterative calculation.

[0008] Optionally, in any mode, at least two camera modules of the defocused 3D camera are working on the object under test.

[0009] Optionally, at least two sets of image information are synchronously acquired two-dimensional image data. Based on the defocus information of at least two sets of camera modules, the depth measurement value of the object to be measured is obtained, including: acquiring images of the two-dimensional data of at least two sets of defocused images to obtain the defocus distance of at least two sets of defocused images; and obtaining the depth of the object to be measured through an optimization algorithm based on the at least two sets of defocused images and defocus distance.

[0010] In this invention, the beam splitting device in the defocused 3D camera splits the incident light into two groups, which are then projected into the camera module. At least two camera modules are used to acquire a two-dimensional defocused image and defocus distance of the object under test in the same scene. The control device processes the defocused image and defocus distance using an algorithm, and by optimizing the calculation, obtains the depth information of the object under test. This achieves the goal of completing image acquisition and three-dimensional depth measurement in a short time, achieving the technical effect of quickly and accurately obtaining the depth value of the object under test with fewer images. This solves the problems of relying on multiple defocused shots, long time consumption, and high measurement costs in related depth measurement methods. Attached Figure Description

[0011] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0012] Figure 1 This is a flowchart illustrating the operation of an off-focus 3D camera according to an embodiment of this application.

[0013] Figure 2 This is a schematic diagram of the calculation process of the defocused 3D camera depth measurement method provided in the embodiments of this application;

[0014] Figure 3 This is a schematic diagram of the operation flow of the defocused 3D camera depth measurement method provided in the embodiments of this application;

[0015] The above figures include the following reference numerals:

[0016] 1. Object to be measured; 2. Beam splitter; 3a. Lens group 1; 3b. Lens group 2; 4a. Range measuring device 1; 4b. Range measuring device 2; 5a. Photosensitive device 1; 5b. Photosensitive device 2; 6. Control device. Detailed Implementation

[0017] The present invention will be further described below with reference to embodiments, but it should not be construed that the scope of the present invention is limited to the following embodiments. Various substitutions and modifications made based on ordinary technical knowledge and common practices in the art without departing from the above-described technical concept of the present invention should be included within the scope of protection of the present invention.

[0018] All other embodiments obtained by a person skilled in the art without creative effort shall fall within the scope of protection of this application.

[0019] According to an embodiment of this application, a defocused 3D camera is provided.

[0020] Figure 1 This is a flowchart illustrating the operation of an off-focus 3D camera according to an embodiment of this application. Figure 1 As shown, the defocused 3D camera includes: a beam splitter, a camera module, and a control device.

[0021] A beam splitter is used to split the incident light rays from the object under test into two sets of optical paths.

[0022] A camera module is used to receive the light path projected by the object under test and to acquire defocused images, and the camera module includes at least two sets of devices.

[0023] The control device is used to determine the working mode of the defocused 3D camera, acquire the defocused image of the object to be measured from the photosensitive device in the camera module and acquire the defocus distance from the ranging device, take the dual defocused image and the defocus distance as input, and obtain the depth measurement information of the object to be measured through a depth optimization algorithm.

[0024] Optionally, in the defocused 3D camera provided in this application embodiment, the beam splitting device can be any of the following: one or more flat beam splitters, one or more cubic beam splitters. The beam splitting device can divide the incident light rays from the scene into two or more groups of light paths according to a given angle and a specified ratio.

[0025] Optionally, such as Figure 2As shown, in the defocused 3D camera provided in this application embodiment, when there is only one beam splitter, the camera module is disposed on the beam splitting transmission path emitted from the beam splitter, and the camera module includes a lens group, a ranging device, and a photosensitive device. The lens group is located in two optical paths and is used to converge the two sets of light rays separated by the beam splitter onto two photosensitive imaging devices; the ranging device has two sets of ranging devices for measuring the distance from the optical center of the lens group to the photosensitive imaging device; and the photosensitive device has two sets of photosensitive imaging elements for photosensitive imaging.

[0026] This application also provides a method for depth measurement using an off-focus 3D camera. It should be noted that the off-focus 3D camera used in this application can be used to execute the depth measurement method provided in this application. The following describes the depth measurement method provided in this application.

[0027] Figure 3 This is a schematic diagram of the operation flow of the defocused 3D camera depth measurement method according to an embodiment of this application, as shown below. Figure 3 As shown, the method includes the following steps:

[0028] Step 1: Arrange the relative positions of the beam splitter and the camera module, control the light transmission of the two beam paths split by the beam splitter to be consistent, and collect a dual defocus image of the same scene through the beam splitter and the camera module.

[0029] Step 2: The control device performs blind deconvolution on the out-of-focus image acquired by the camera module's photosensitive device to obtain a clear scene image. The clear scene image is then convolved with the out-of-focus distance and initial depth obtained by the ranging device to obtain a reconstructed out-of-focus image.

[0030] Step 3: Perform initial depth optimization calculations based on the difference loss between the reconstructed out-of-focus image and the out-of-focus image acquired by the camera module to obtain the depth value of the object to be measured.

[0031] Optionally, in the defocused 3D camera depth measurement method provided in the embodiments of this application, the blind deconvolution operation method includes, but is not limited to, any one of the following: inverse filtering algorithm, regularization algorithm.

[0032] Optionally, in the defocused 3D camera depth measurement method provided in the embodiments of this application, the convolution synthesis operation method includes, but is not limited to, convolution operation based on point spread function.

[0033] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0034] The above are merely embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the principles of this application should be included within the scope of the claims of this application.

Claims

1. A defocused 3D camera, characterized in that, include: A beam splitter is used to split the incident light rays from the object to be measured into two sets of light paths with the same output light. The camera module includes at least two sets of devices, which are set on the optical path transmission path of the beam splitter. After the incident light rays of the optical path pass through the camera module, a dual defocus map is generated to collect the image information of the scene to be tested. The camera module includes a lens group, a ranging device, and a photosensitive device; the lens group, located in the optical path, is used to converge the light rays separated by the beam splitter onto the photosensitive device; the ranging device is used to measure the defocus distance from the optical center of the lens to the photosensitive imaging device; and the photosensitive device has a photosensitive imaging element located at the end of the camera device for photosensitive imaging. The control device is used to determine the working mode of the defocused 3D camera, perform blind deconvolution operation on the dual defocused images to obtain a scene clear image, perform convolution operation on the given initial depth, scene clear image and defocus distance to obtain a reconstructed defocused image, and perform accurate depth calculation by the difference between the reconstructed defocused image and the actual captured defocused image.

2. The defocused 3D camera according to claim 1, characterized in that, The beam splitter further includes any one of the following: One or more flat beam splitters divide the incident light rays into two or more desired optical paths according to a given angle and a specified ratio. One or more cubic beam splitters divide the incident light rays into two or more desired light paths according to a given angle and a specified ratio.

3. The camera module according to claim 1, characterized in that, The photosensitive plate of the photosensitive device is of CCD or CMOS type.

4. The camera module according to claim 1, characterized in that, The ranging device further includes any one of the following: A laser ranging device is used to obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the defocus depth during image acquisition. An infrared ranging device is used to obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the defocus depth during image acquisition.

5. The defocused 3D camera according to claim 1, characterized in that, The control device is specifically used for: Control the working state of the camera module; The camera module is controlled to form an image, and the defocus image obtained by the photosensitive device and the defocus distance obtained by the ranging device are acquired. The obtained defocus map and defocus distance are input into the depth calculation algorithm to obtain the depth measurement result.

6. A method for acquiring images using a defocused 3D camera, characterized in that, The data acquisition method includes: The incident light rays of the object under test are split by a beam splitter and projected onto the camera module in the form of two or more optical paths. At least two camera modules acquire image information of the object under test at different defocus distances. The image information consists of at least two sets of image information. The defocused 3D camera image acquisition method is performed by the defocused 3D camera as described in claim 1.

7. A method for depth measurement using a defocused 3D camera, characterized in that, The depth measurement method includes: The incident light from the object under test is projected onto the camera module through a beam splitter. The control device controls the photosensitive device of the camera module to form an image and acquires the defocus image obtained by the photosensitive device and the defocus distance obtained by the ranging device. The defocused image is processed by a blind deconvolution algorithm to obtain a clear image of the corresponding scene. The clear image is combined with the given initial scene depth and the defocus depth obtained by the ranging device to generate a corresponding reconstructed defocused image through a convolution synthesis algorithm. The reconstructed defocus image is compared with the real defocus image acquired by the camera under the corresponding parameters, the difference loss value between the images is calculated, and the initial scene depth is updated and optimized based on the loss value, so as to obtain an accurate scene depth value after optimization. The defocused 3D camera depth measurement method is performed by the defocused 3D camera as described in claim 1.

8. A storage medium, characterized in that, The storage medium includes a stored program, wherein the program is executed by a computer system to implement the method of any one of claims 6 or 7.

9. A processor, characterized in that, The processor is used to run a program that performs the method described in any one of claims 6 or 7.

Images