Light-path-splitting three-dimensional measurement system

Abstract

The invention discloses a light-path-splitting three-dimensional measurement system, and relates to the technical field of camera imaging and depth measurement. Most of existing photogrammetric systems cannot collect two or more pictures of the same scene and under the same view angle and calculate dense scene depth at the same time, the cost for obtaining the two or more pictures of the same scene and under the same view angle is high, the obtained depth is sparse, the precision is not high, and the problem can be effectively solved with low cost. The system comprises a light splitting path acquisition device, a storage calculation device and a depth measurement method. Wherein the light splitting path acquisition device comprises a spectroscope device, a lens group, an aperture, a focusing device, a distance measuring device, a photosensitive imaging device and a calculation control device, and two or more groups of light paths are acquired by utilizing the spectroscope device, so that a clear image, a fuzzy image and a focusing distance under the same visual angle are acquired at the same time. And the storage and calculation device calculates the scene depth by using the data acquired by the light splitting path acquisition device according to a corresponding depth measurement method.

Description

Technical Field

[0001] The present application relates to the technical field of camera imaging and depth measurement, and in particular to a three-dimensional measurement system for split light paths. Background Art

[0002] The split-beam path three-dimensional measurement system is a measurement system that uses a beam splitter device to split the scene input light into two, obtains a clear image and a blurred image of the same scene through two sets of light paths, and uses the corresponding depth measurement method to calculate the scene depth. The current depth measurement systems mainly include lidar systems, structured light systems, and binocular stereo vision systems. The lidar system is expensive and the acquired depth is sparse. The structured light system equipment is complex and requires active projection, and it is unable to calculate the depth of the reflective and translucent parts. The binocular stereo vision uses two cameras to obtain two pictures from different perspectives, and restores the scene depth by calculating the parallax between the two pictures. It requires complex calibration and matching, has occlusion problems, and the acquired depth is sparse.

[0003] In the existing monocular vision technology, clear images or focus stacks are mainly used to obtain scene depth through neural networks or traditional algorithms. However, the depth obtained using clear images is often relative depth and has poor accuracy. The method of obtaining depth using focus stacks is complicated to operate and requires obtaining a large number of pictures with different focus depths in the same scene, which has poor practicality. The blurred image has strong depth information, and the depth information in the blurred image can be used to obtain more accurate scene depth. At present, the method of using blurred images to obtain depth has been widely used.

[0004] The method of obtaining depth using a clear image or a focus stack has problems such as low accuracy or complex implementation. The method of obtaining depth using a blur image and a clear image combined with a point spread function can effectively solve this problem. Most of the current photogrammetry systems cannot simultaneously obtain two or more images of the same scene and the same viewing angle and calculate the dense scene depth. The cost of obtaining two or more images of the same scene and the same viewing angle is high, and the obtained depth is sparse and the accuracy is low. The split-light path three-dimensional measurement system can effectively solve this problem at a low cost. Summary of the invention

[0005] The present application provides a split-light path three-dimensional measurement system to solve the problem that the existing photography system cannot simultaneously obtain two pictures of the same scene with the same perspective at a low cost, and the method of obtaining depth using clear pictures or focus stacks is not accurate or complex to implement.

[0006] According to another aspect of the present application, a split light path acquisition device is provided, which includes a beam splitter device, two lens groups, an aperture, a focusing device, two sets of distance measuring devices, two photosensitive imaging devices and a computing control device. The beam splitter device is used to split the light incident from the shooting scene into two groups of light paths; the two lens groups are respectively located in the two groups of light paths, and are used to converge the two groups of light separated by the beam splitter onto two photosensitive imaging devices; the aperture is located behind the lens group, and is used to adjust the light transmission of the lens to adjust the depth of field of the shooting; the two sets of focusing devices are respectively located in the two groups of light paths, between the lens group and the photosensitive imaging device, and are used to adjust the distance between the optical center of the lens group and the photosensitive imaging device, so as to adjust the focusing distance during shooting; the two sets of distance measuring devices are used to measure the distance from the optical center of the lens group in the two groups of light paths to the photosensitive imaging device, so as to obtain the focusing distance of the two pictures; the two sets of photosensitive imaging devices are used for scene imaging, and obtain clear images and blurred images under the same viewing angle of the same scene; the calculation control device is used to control the two sets of photosensitive imaging devices to image, obtain corresponding clear images and blurred images, compare and calculate the brightness difference between the clear images and the blurred images, adjust the exposure time of the photosensitive imaging device, and control the two sets of photosensitive imaging devices to image again, so as to obtain clear images and blurred images with consistent brightness.

[0007] Optionally, the spectroscope device also includes any one of the following: a flat-plate spectroscope, which can split the incident light of the scene into two according to a given angle and a specified ratio to obtain two sets of required light paths; a cube spectroscope, which can split the incident light of the scene into two according to a given angle and a specified ratio to obtain two sets of required light paths.

[0008] Optionally, the beam splitter device further comprises any one of the following: the beam splitter device splits the incident light of the scene into two groups of light paths to obtain two pictures of the same scene. The beam splitter device splits the incident light of the scene into multiple groups of light paths to obtain multiple pictures of the same scene.

[0009] Optionally, the aperture can be adjusted arbitrarily within an adjustable range, and can individually adjust the lens light transmittance in two groups of light paths and adjust the shooting depth of field, so that the photosensitive imaging device under each group of light paths can obtain corresponding clear images or blurred images with different depths of field.

[0010] Optionally, the focusing device also includes any one of the following: one set of focusing devices, which is located in two groups of light paths at the same time, and can adjust the distance between the optical center of the lens group and the photosensitive imaging device in the two groups of light paths respectively, thereby adjusting the focusing distance of the two pictures when shooting; two sets of focusing devices, which are respectively located in the two groups of light paths, and can individually adjust the distance between the optical center of the lens group and the photosensitive imaging device in the two groups of light paths, thereby adjusting the focusing distance of the two pictures when shooting.

[0011] Optionally, the distance measuring device also includes any one of the following: a laser distance measuring device capable of obtaining the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device; an infrared distance measuring device capable of obtaining the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device.

[0012] Optionally, the calculation and control device is specifically used to: compare and calculate the brightness difference between the pictures obtained by the two sets of photosensitive imaging devices; adjust the exposure time of the photosensitive imaging device according to the brightness difference between the two pictures; control the imaging of the two sets of photosensitive imaging devices to obtain two pictures with consistent brightness.

[0013] According to another aspect of the present application, a depth measurement method is provided. The method includes: using the light splitting path acquisition device to collect a clear image, a blur image, and a focus distance of the blur image of the same scene and the same viewing angle with the same brightness; the clear image is combined with a given initial scene depth and the focus depth obtained by the distance measuring device to generate a corresponding blur image through a point spread function; the generated blur image is compared with a real blur image obtained by the camera under corresponding parameters; the initial scene depth is optimized according to the difference between the generated blur image and the real blur image, and the accurate scene depth is obtained after optimization.

[0014] According to another aspect of the present application, a three-dimensional measurement system for a split light path is provided. The three-dimensional measurement system for a split light path comprises: a split light path acquisition device, which is the split light path acquisition device of any one of claims 2 to 9; a storage and calculation device, which is used to store data such as a clear image, a blurred image, and a focus distance collected by the split light path acquisition device, calculate the scene depth using the depth measurement method, and store and display the depth. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 It is a schematic diagram of a light splitting path acquisition device provided according to an embodiment of the present application.

[0017] Figure 2 It is a schematic diagram of a three-dimensional measurement system for a light splitting path provided according to an embodiment of the present application.

[0018] Figure 3 is a schematic diagram of an optional light path collection device provided according to an embodiment of the present application Figure 1 .

[0019] Figure 4 is a schematic diagram of an optional light path collection device provided according to an embodiment of the present application Figure 2 .

[0020] Figure 5 This is a depth measurement method provided according to an embodiment of the present application.

[0021] Figure 6 It is a schematic diagram of an optional light splitting path three-dimensional measurement system provided according to an embodiment of the present application.

[0022] The drawings in the appeal include the following reference numerals:

[0023] 1. Spectroscope device; 2a. Lens group 1; 2b. Lens group 2; 3a. Aperture 1; 3b. Aperture 2; 4a. Distance measuring device 1; 4b. Distance measuring device 2; 5a. Focusing device 1; 5b. Focusing device 2; 5c. Focusing device 3; 6a. Photosensitive imaging device 1; 6b. Photosensitive imaging device 2; 7. Computing and controlling device; 8. Storage and computing device. DETAILED DESCRIPTION

[0024] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. In order to make the purpose, technical solutions and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0025] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0026] The terms "1", "2", etc. in the specification and drawings of the present application are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence, unless otherwise specified.

[0027] According to an embodiment of the present application, a split-light path acquisition device is provided.

[0028] Figure 1 Schematic diagram of a light path acquisition device according to an embodiment of the present application. Figure 1 As shown, the light splitting path acquisition device includes: a spectroscope device, a lens group and an aperture, a focusing and ranging device, a photosensitive imaging device and a computing and control device.

[0029] The beam splitter device is used to divide the incident light from the scene into two or more light paths according to a given angle and a specified ratio.

[0030] The lens group and the aperture are arranged in each light path, wherein the lens group is used to converge the image, and the aperture is used to adjust the amount of light passing through each light path and the depth of field of the shooting, so that the acquisition device can obtain clear and blurred images of the scene.

[0031] The focusing and ranging device is arranged in each optical path and is used to adjust and measure the focusing depth of the acquisition device.

[0032] The photosensitive imaging device and the computing control device are arranged in each optical path to adjust the exposure time of the acquisition system and the scene imaging.

[0033] In an optional example, the beam splitter device may be a flat beam splitter or a cube beam splitter, which can split the incident light of the scene into two or more groups of light paths according to a given angle and a specified ratio.

[0034] In an optional example, the aperture in each group of light paths can be adjusted arbitrarily within an adjustable range to obtain a corresponding clear image of the shooting scene or a blurred image with different depths of field.

[0035] In an optional example, when the beam splitter device divides the incident light of the scene into two groups of light paths according to a given angle and a specified ratio, the focusing device can be one set or two sets. When the focusing device is one set, the focusing device is simultaneously located in two or more groups of light paths, and can adjust the distance between the optical center of the lens group and the photosensitive imaging device in each group of light paths; when the focusing device is two sets, the focusing device is respectively located in two groups of light paths, and can adjust the distance between the optical center of the lens group and the photosensitive imaging device in each group of light paths separately;

[0036] In an optional example, the distance measuring device may be a laser distance measuring device or an infrared distance measuring device, which can obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device.

[0037] In an optional example, the photosensitive chip in the photosensitive imaging device may be a CCD or a CMOS.

[0038] Figure 3 Schematic diagram of an optional light path acquisition device provided according to an embodiment of the present application. Figure 3 As shown, the light-split path acquisition device includes a beam splitter device, a lens group 1, a lens group 2, an aperture 1, an aperture 2, a focusing device 1, a focusing device 2, a distance measuring device 1, a distance measuring device 2, a photosensitive imaging device 1, a photosensitive imaging device 2, and a computing control device. In the light-split path acquisition device, the beam splitter device can be a plane beam splitter or a stereo beam splitter, which can divide the scene incident light into two or more groups of light paths. The schematic diagram of the light-split path acquisition device takes two light paths as an example for a clear and complete description.

[0039] The collection process of the light splitting path collection device is as follows:

[0040] The beam splitter device splits the light incident from the shooting scene into two to obtain light path 1 and light path 2;

[0041] The lens group 1 and the lens group 2 behind the beam splitter device are respectively located in the optical path 1 and the optical path 2, and the light in the two optical paths is respectively converged into the photosensitive imaging device 1 and the photosensitive imaging device 2;

[0042] The aperture behind the lens group controls the amount of light passing through the lens. Aperture 1 in optical path 1 is adjusted to the maximum to minimize the depth of field and obtain a blurred image of the scene. Aperture 2 in optical path 2 is adjusted to the minimum to maximize the depth of field and obtain a clear image of the scene.

[0043] Adjust the focusing device 1 in the optical path 1 so that the image collected by the optical path 1 is focused at an appropriate depth to obtain a suitable blurred image, and adjust the focusing device 2 in the optical path 2 so that the image collected by the optical path 2 is focused at an appropriate position to obtain a clear image of the shooting scene;

[0044] Distance measuring device 1 and distance measuring device 2 respectively measure the distance from the optical center of the lens group in optical path 1 and optical path 2 to the photosensitive imaging device to obtain the focus distance of the pictures taken in the two optical paths;

[0045] The computing control device controls the photosensitive imaging device 1 in the optical path 1 and the photosensitive imaging device 2 in the optical path 2 to perform photosensitive imaging, and obtains a clear image and a blurred image corresponding to the shooting scene;

[0046] The computing and controlling device compares the comprehensive brightness difference between the two groups of pictures acquired by the photosensitive imaging device 1 and the photosensitive imaging device 2;

[0047] The calculation control device adjusts the exposure time of the photosensitive imaging device in the optical path 2 to adjust the brightness of the clear image obtained by the optical path 2;

[0048] The computing control device controls the photosensitive imaging device 1 and the photosensitive imaging device 2 to perform photosensitive imaging again, so as to obtain a clear image and a blurred image with consistent brightness under the same viewing angle.

[0049] Optionally, the lens group 1 and the lens group 2 may be the same or different, and the aperture 1 or the aperture 2 may be the same or different.

[0050] Optionally, the focusing device 1 and the focusing device 2 may be the same or different.

[0051] Optionally, the distance measuring device 1 and the distance measuring device 2 may both be laser distance measuring devices, both be infrared distance measuring devices, or a combination of a laser distance measuring device and an infrared distance measuring device.

[0052] The optional photosensitive imaging device 1 and the photosensitive imaging device 2 may both be CCD chips, both be CMOS chips, or a combination of CCD and CMOS.

[0053] The spectroscopic path acquisition device provided in the embodiment of the present application simultaneously acquires a clear image and a blurred image of the same scene and the same viewing angle through image acquisition devices such as a spectroscope device and a lens group, thereby solving the problem that it is difficult for existing photographic acquisition devices to simultaneously acquire a clear image, a blurred image and a focus distance of the same scene and the same viewing angle, and the cost of existing photographic acquisition devices to acquire a clear image, a blurred image and a focus distance of the same scene and the same viewing angle is relatively high.

[0054] At this time, the split-light path acquisition device also has the following features: 1. Low hardware cost; 2. Miniaturization of equipment; 3. Easy operation; 4. Fast acquisition speed. This solves the problem that the existing photographic acquisition device is difficult to simultaneously obtain a clear image, a blurred image, and a focus distance at the same scene and the same viewing angle, and the existing photographic acquisition device has a high cost for obtaining a clear image, a blurred image, and a focus distance at the same scene and the same viewing angle.

[0055] Figure 4 is a schematic diagram of another optional split-light path collection device provided according to an embodiment of the present application, and its working principle and required devices are similar to Figure 3 The light path collection device shown is similar, the main difference is Figure 4 The split light path acquisition device has only one focusing device, which is located in both light paths and can adjust the distance from the optical center of the lens group in the two light paths to the photosensitive imaging device, and adjust the focusing distance of the two light paths.

[0056] The present application also provides a depth measurement method. It should be noted that the light splitting path three-dimensional measurement system of the present application can be used to perform the depth measurement method provided by the present application. The depth measurement method provided by the present application is introduced below.

[0057] Figure 5 is a schematic diagram of a depth measurement method according to an embodiment of the present application. Figure 5 As shown in FIG. 1 , the depth measurement method uses a clear image, a blurry image, and the focus distance of the blurry image to obtain the scene depth. The specific implementation process is as follows:

[0058] Arbitrarily give an initial scene depth;

[0059] The clear image is combined with the initial scene depth and focus distance to generate the corresponding blur image through the PSF camera model;

[0060] The generated fuzzy image is compared and calculated with the real fuzzy image;

[0061] Optimizing and updating the initial depth according to the comparison calculation result between the generated blur map and the real blur map;

[0062] Get a more accurate scene depth.

[0063] The depth measurement method provided in the embodiment of the present application uses a clear image and a blurred image to obtain the scene depth through simple calculation, which solves the problem of low accuracy or complex implementation of the method of obtaining depth using a clear image or a focus stack.

[0064] The present application also provides a three-dimensional measurement system for a split light path. Figure 2 , Figure 6 Schematic diagram of a three-dimensional measurement system for light splitting paths according to an embodiment of the present application.

[0065] like Figure 6 As shown, the light splitting path three-dimensional measurement system includes a light splitting path acquisition device and a storage and calculation device, and its specific implementation process is as follows:

[0066] The light-splitting acquisition device acquires the clear image, blurry image and focus distance of the same scene and the same viewing angle;

[0067] The data collected by the light splitting path collection device is stored in the storage and calculation device;

[0068] The storage and calculation device calculates the scene depth using the depth measurement method;

[0069] The storage and calculation device displays and stores the scene depth.

[0070] It should be noted that the computing control device in the storage computing device and the light splitting path acquisition device includes any one of the following:

[0071] Two separate devices, each performing its own task independently;

[0072] The same device can perform the two tasks described above respectively.

[0073] It should be noted that the light splitting path collection device can be any of the light splitting path collection devices mentioned above.

[0074] The storage computing device may be a computer, a server, a PC, a mobile phone, etc.

[0075] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A three-dimensional measurement system for split light paths, characterized in that: The light splitting path three-dimensional measurement system comprises: A light path acquisition device, used to acquire a clear image, a blurred image and a corresponding focus distance of a scene; The storage and calculation device is used to store the data such as the clear image, blurred image and focus distance collected by the light path collection device, calculate the scene depth using the corresponding depth measurement method, and store and display the depth.

2. A light path collection device, characterized in that: The light splitting path collection device comprises: A beam splitter device, used to split the light incident from the shooting scene into two groups of light paths; The lens group, the light path collection device has two lens groups, which are located in the two light paths and are used to converge the two groups of light separated by the beam splitter device onto the two photosensitive imaging devices; Aperture: the light path acquisition device has two apertures, which are located behind the two lens groups and are used to adjust the light throughput of the lens and the depth of field of the shooting; The focusing device is located in the two optical paths and is used to adjust the distance between the optical center of the lens group in the two optical paths and the photosensitive imaging device, thereby adjusting the focus distance during shooting; Distance measuring device, the light path collection device has two sets of distance measuring devices for measuring the distance from the optical center of the lens group to the photosensitive imaging device; Photosensitive imaging device, the split light path acquisition device has two sets of photosensitive imaging devices, located in two groups of light paths, for scene imaging; The computing control device includes a computing control device for adjusting the exposure time of the two sets of photosensitive imaging devices and controlling the photosensitive imaging of the photosensitive imaging devices.

3. The light path acquisition device according to claim 2, characterized in that: The spectroscopic device further comprises any one of the following: The flat beam splitter can split the incident light from the scene into two according to a given angle and a specified ratio, thus obtaining two sets of required light paths; The cube beam splitter can split the incident light from the scene into two according to a given angle and specified ratio, obtaining two sets of required light paths.

4. The light path acquisition device according to claim 2, characterized in that: The spectroscopic device further comprises any one of the following: The beam splitter device divides the incident light of the scene into two groups of light paths to obtain two pictures of the same scene. The beam splitter device divides the incident light of the scene into multiple groups of light paths to obtain multiple pictures of the same scene.

5. The light path acquisition device according to claim 2, characterized in that: The aperture is located behind the lens group and can be adjusted arbitrarily within an adjustable range. It can individually adjust the lens light transmittance in the two light paths and adjust the shooting depth of field, so that the photosensitive imaging device in each light path can obtain a corresponding clear image or a blurred image with different depths of field.

6. The light path acquisition device according to claim 2, characterized in that: The focusing device also includes any one of the following: A set of focusing devices, located in both optical paths, can adjust the distance between the optical center of the lens group and the photosensitive imaging device in the two optical paths respectively, thereby adjusting the focus distance of the two pictures when shooting; Two sets of focusing devices are located in the two optical paths respectively, and can individually adjust the distance between the optical center of the lens group and the photosensitive imaging device in the two optical paths, thereby adjusting the focus distance of the two pictures when shooting.

7. The light path acquisition device according to claim 2, characterized in that: The distance measuring device further includes any one of the following: The laser distance measuring device can obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the focus depth of the image; The infrared distance measuring device can obtain the distance from the optical center of the lens group to the photosensitive sensor in the photosensitive imaging device, thereby obtaining the focus depth of the image.

8. The light path acquisition device according to claim 2, characterized in that: The photosensitive chip in the photosensitive imaging device may be a CCD or a CMOS, and the two sets of photosensitive imaging devices are specifically used for: Perform a second separate imaging of the same scene to obtain two different pictures of the same scene; After the aperture is reduced, the exposure time is adjusted so that the overall brightness of the two collected images is close to the same.

9. The light path acquisition device according to claim 2, characterized in that: The computing control device is specifically used to: control the two sets of photosensitive imaging devices to form images, and obtain corresponding clear images and blurred images; Comparing and calculating the brightness difference between the clear image and the blurred image; Adjusting the exposure time of the photosensitive imaging device; The two sets of photosensitive imaging devices are controlled to image again to obtain a clear image and a blurred image with consistent brightness.

10. A depth measurement method, characterized in that: The depth measurement method comprises: Using the acquisition device to acquire a clear image and a blurred image with the same brightness and myopia in the same scene and the same viewing angle, and to acquire the focus distance of the blurred image; The clear image is combined with a given initial scene depth and a focus depth obtained by a ranging device to generate a corresponding blur image through a point spread function; The generated blur image is compared and calculated with the real blur image collected by the camera under corresponding parameters; The initial scene depth is optimized according to the difference between the generated blur map and the real blur map, and an accurate scene depth is obtained after optimization.

Images