Binocular vision ranging method based on multi-ocular micro super lens

Abstract

The invention relates to a binocular vision distance measurement method based on a multi-eye micro-super lens. The method comprises the following steps that a measured object, a telescope structure, the multi-eye micro-super lens and an optical detector are sequentially arranged in the light path direction; acquiring a multi-view detection image of the detected object at different set positions as a calibration image set; a pair of sub-eyes of the multi-eye micro-super lens is selected to form a binocular ranging structure; fitting a function relationship between the distance from the measured object to the telescope structure and the imaging parallax value of the binocular ranging structure based on the calibration image set; and acquiring a multi-view detection image of a detected object at an unknown position, analyzing to obtain an imaging parallax value of the binocular ranging structure, and substituting the imaging parallax value into the function relation to calculate to obtain a distance prediction value from the current position of the detected object to the telescope structure. According to the invention, the sub-eye pairs of the multi-eye micro super lens are adopted to replace a traditional binocular vision device and are optimized, so that the calibration process is simplified while the measurement precision is ensured, and the limitation on the physical structure of the traditional binocular vision is broken through.

Description

Technical Field

[0001] The present invention relates to the technical field of optical measurement, and particularly to a binocular vision ranging method based on a multi-eye microlens. Background Art

[0002] The passive binocular depth camera is based on the binocular vision principle and uses the parallax of the images formed by two cameras to calculate depth / distance information. Existing products mainly use binocular cameras, which have a large structural volume, a complex calibration process, and can hardly expand the working distance. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a binocular vision ranging method based on a multi-eye microlens, which can reduce the volume of the measuring device and simplify the calibration process while ensuring the measurement accuracy.

[0004] The technical solution adopted by the present invention to solve its technical problems is: providing a binocular vision ranging method based on a multi-eye microlens, including the following steps:

[0005] A measured object, a telescope structure, a multi-eye microlens, and an optical detector are sequentially arranged along the optical path direction;

[0006] Obtaining multi-eye detection images of the measured object at different set positions as a calibration image set, where the multi-eye detection images include multiple imaging images collected by the optical detector after the measured object at the current position is imaged by each sub-eye of the multi-eye microlens respectively;

[0007] Selecting a pair of sub-eyes of the multi-eye microlens to form a binocular ranging structure;

[0008] Based on the calibration image set, fitting out the functional relationship between the distance from the measured object to the telescope structure and the imaging parallax value of the binocular ranging structure;

[0009] Collecting multi-eye detection images of the measured object at an unknown position, analyzing to obtain the imaging parallax value of the binocular ranging structure, and substituting it into the functional relationship to calculate the predicted value of the distance from the current position of the measured object to the telescope structure.

[0010] Further, fitting out the functional relationship between the distance from the measured object to the telescope structure and the imaging parallax value of the first pair of sub-eyes based on the calibration image set, including:

[0011] Obtaining the baseline distance of the first pair of sub-eyes;

[0012] Establishing a mathematical model between the distance from the measured object to the telescope structure, the baseline distance of the first pair of sub-eyes, and its imaging parallax value;

[0013] Based on the multi-view detection images of the object to be measured at each set position, calculate the imaging parallax of the first sub-eye pair at the corresponding position, and then use an optimization algorithm to fit the optimal parameters of the mathematical model to obtain the functional relationship.

[0014] Further, the mathematical model is expressed as

[0015]

[0016] where Depth is the distance from the object to be measured to the telescope structure, B is the baseline distance between a pair of selected sub-eyes, d is the imaging parallax of the pair of selected sub-eyes, and p1, p2, and p3 are model parameters.

[0017] Further, it also includes selecting several different sub-eye pairs to obtain multiple binocular ranging structures, calculating the distance prediction values corresponding to each binocular ranging structure according to the multi-view detection images, and then calculating the distance measurement value as the average of all distance prediction values.

[0018] Further, the optical detector simultaneously collects the imaging images of all sub-eyes of the multi-view microlens array.

[0019] Further, the telescope structure is installed on the multi-view microlens array so that all sub-eyes of the multi-view microlens array share the same optical path as the telescope structure.

[0020] Further, the telescope structure includes an objective lens and an eyepiece. After the object to be measured is imaged by the objective lens and the eyepiece of the telescope structure in sequence, it is imaged on the optical detector by each sub-eye of the multi-view microlens array respectively.

[0021] Advantageous Effects

[0022] Due to the adoption of the above technical solutions, compared with the prior art, the present invention has the following advantages and positive effects: The present invention constructs a binocular system by using a planar multi-view microlens array, greatly reducing the volume of the binocular vision system. At the same time, due to the planar nature of the planar multi-view microlens array itself, the calibration process becomes simple, effectively avoiding the complex calibration process caused by different planes and different angles of binocular cameras in the prior art; The present invention installs a telescope on the multi-view microlens array, breaking through the limitation of difficult working distance extension in the prior art and effectively extending the working distance; The present invention utilizes the multi-view characteristics of the microlens array, measures the prediction values by selecting multiple different sub-eye pairs and takes the average, solving the problem that the measurement error of parallax leads to an increase in the prediction depth error as the depth increases, making the measurement result, especially the measurement result at a long distance, more accurate, and further improving the working distance. Description of the Drawings

[0023] Figure 1 is a flowchart of an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of the optical path when a telescope is installed and not installed in an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of the system in an embodiment of the present invention;

[0026] Figure 4 is a depth-parallax relationship diagram of a single pair of sub-eyes when a telescope is installed in an embodiment of the present invention;

[0027] Figure 5 is a multi-eye detection diagram when a telescope is not installed in an embodiment of the present invention;

[0028] Figure 6 is an imaging diagram of two selected sub-eyes when a telescope is not installed in an embodiment of the present invention;

[0029] Figure 7 is a parallax diagram and a predicted depth diagram when a telescope is not installed in an embodiment of the present invention;

[0030] Figure 8 is a depth-parallax relationship diagram of a single pair of sub-eyes when a telescope is not installed in an embodiment of the present invention;

[0031] Figure 9 is a schematic diagram of the predicted depth fluctuation of multiple pairs of sub-eyes when a telescope is not installed in an embodiment of the present invention;

[0032] Figure 10 is a multi-eye detection diagram when a telescope is installed in an embodiment of the present invention;

[0033] Figure 11 is an imaging diagram of two selected sub-eyes when a telescope is installed in an embodiment of the present invention;

[0034] Figure 12 is a parallax diagram and a predicted depth diagram when a telescope is installed in an embodiment of the present invention;

[0035] Figure 13 is a schematic diagram of the predicted depth fluctuation when a telescope is installed in an embodiment of the present invention. Specific Embodiments

[0036] The following further elaborates the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of this application.

[0037] The embodiment of the present invention relates to a binocular vision ranging method based on a multi-eye micro-superlens, asFigure 1 As shown, it includes the following steps:

[0038] An object to be measured, a telescope structure, a multi-eye micro-superlens, and an optical detector are sequentially arranged along the optical path direction;

[0039] Obtain multi-eye detection images of the object to be measured at different set positions as a calibration image set. The multi-eye detection images include multiple imaging images collected by the optical detector after the object to be measured at the current position is respectively imaged by each sub-eye of the multi-eye micro-superlens;

[0040] Select a pair of sub-eyes of the multi-eye micro-superlens to form a binocular ranging structure;

[0041] Based on the calibration image set, fit the functional relationship between the distance from the object to be measured to the telescope structure and the imaging parallax value of the binocular ranging structure;

[0042] Collect the multi-eye detection images of the object to be measured at an unknown position, analyze to obtain the imaging parallax value of the binocular ranging structure, and substitute it into the above functional relationship to calculate the predicted value of the distance from the current position of the object to be measured to the telescope structure.

[0043] The planar multi-eye superlens is a device based on metasurface technology, with the ability of multi-field-of-view perception and independent imaging of each sub-eye. Any pair of sub-eyes can form a parallel binocular vision structure and has the ability to sense distance. Due to the compact size of the planar multi-eye superlens, the baseline distance of the sub-eye pair is generally at the millimeter level (the traditional binocular vision is at the centimeter level), and all sub-eyes can be imaged by one detector.

[0044] Generally speaking, the ranging range of the binocular system is positively correlated with the binocular baseline distance and the focal length of the imaging system. The baseline distance of conventional binocular camera products is at the centimeter level. For example, for the product Gemini2 of Orbbec, the baseline distance is 5 cm, and its measurement distance range is only within several meters. Therefore, when using a multi-eye micro-superlens to replace the traditional parallel binocular vision measurement device, the problem of too small measurement range needs to be solved. Thanks to the compact size of the planar multi-eye superlens, a single telescope can be installed to make all sub-eyes share the same optical path, and the working distance can be greatly extended at the expense of a certain field-of-view angle. As Figure 2 As shown in (a) and 2(b), taking a pair of sub-eyes as an example, they are the ideal optical path diagrams without and with a telescope installed respectively. After installing the telescope, the proportion of the common field of view becomes smaller, the parallax value increases, and the working distance increases.

[0045] During measurement, as Figure 3 shown, ideally, the object to be measured is first imaged by the objective lens (real image), then imaged by the eyepiece (virtual image), and finally imaged on the detector by the multi-eye lens structure.

[0046] Assume that the distance of the object to be measured from the objective lens of the telescope is depth, the focal length of the objective lens is F1, the focal length of the eyepiece is F2, the distance between the objective lens and the eyepiece is C1, the distance between the eyepiece and the multi-lens is C2, and the distance between the multi-lens and the detector is the focal length F of the multi-lens. The baseline distance between a group of sub-eyes of the multi-lens is B, and the parallax of the image formed by the object through this group of sub-eyes is d.

[0047] According to the objective lens imaging equation, the eyepiece imaging equation, and the multi-lens imaging equation, the relationship between the distance depth and the parallax value d can be deduced, and the deduced result can be simplified into the following formula:

[0048]

[0049] Among them, p1, p2, and p3 are three parameters related to the distances C1, C2, the focal length F1, and the focal length F2. Since the precise parameters of the actual physical structure are difficult to measure and calculate, several points can be selected within a suitable working range to measure the true depth value depth and calculate the parallax value d respectively. Then, the parameters p1, p2, and p3 can be calculated through an iterative fitting algorithm. Then, using the fitted formula, the parallax value d when the distance of the object to be measured is unknown is calculated and substituted, and the distance prediction value depth can be obtained.

[0050] Assume that in the ideal case where the object is a point light source, the object has only one characteristic point, so the imaging parallax can be directly calculated. It is not difficult to find that the error of the distance prediction value depth is directly related to the error in calculating the parallax.

[0051] As Figure 4 shown, although the object is in an ideal situation, due to distance measurement errors, calculation errors, imaging interference, etc., when the distance is far, the relationship curve between the parallax and the depth has certain fluctuations compared with the measured values. At this time, the depth values predicted by a single pair of sub-eyes (several gray broken lines) will have large deviations due to the errors in calculating the parallax value. Therefore, noise reduction processing is required, and methods such as mean filtering and Kalman filtering can be used.

[0052] The following will further illustrate this embodiment in combination with specific embodiments.

[0053] In Embodiment 1, a planar multi-lens hyperlens without a telescope is used for measurement. The hyperlens includes six sub-eyes, and the true distance of the target object is 250 mm. The multi-lens detection image collected by the detector is as Figure 5 shown, where each sub-eye corresponds to generating an imaging image. Select two of the sub-eyes to form a binocular vision device. The images generated by the two sub-eyes are respectively as Figure 6 (a) and Figure 6 (b) shown, and the calculated parallax maps are respectively as Figure 7 (a) and Figure 7(as shown in (b)). The parallax and depth fitting results of single pair of eyes and multiple pairs of eyes are respectively as Figure 8 and Figure 9 shown. It is not difficult to find that there have been large fluctuations in the system near 0.8m.

[0054] In Example 2, a telescope is installed on the planar multi - eye superlens of the example for measurement. The multi - eye detection images collected by the detector are as Figure 10 shown. Among them, the true distance values of the three bottles on the left, middle and right are approximately 25m, 30m, and 20m respectively. The images generated by two of the sub - eyes are respectively as Figure 11 (a) and Figure 11 (b) shown, and the calculated parallax maps are respectively as Figure 12 (a) and Figure 12 (b) shown.

[0055] The measurement results are as Figure 13 shown. Since the relationship between parallax and depth is similar to an inverse - proportional curve, when the distance is far, the parallax becomes smaller and smaller, and the measurement error of the parallax will be more obviously reflected in the depth. Among them, the gray broken line is the predicted value of each pair of sub - eyes of the binocular measurement system in this embodiment. By comprehensively calculating the predicted values of each pair of sub - eyes and taking the average value, the green dots will be obtained, and they are called the overall predicted value of the system, while the red straight line is the true depth. It can be seen that the fluctuations of the green dots in the figure have become significantly smaller. After integrating the parallax information from different perspectives, the accuracy of the predicted value has been significantly improved, which can be understood as taking the average value of white noise, thereby reducing the noise variance. When measuring an object at a relatively close distance, the predicted results of a single pair of sub - eyes also tend to be accurate. Therefore, integrating the information of different pairs of sub - eyes not only makes the measurement results more accurate but also further increases the working distance.

Claims

1. A binocular vision ranging method based on a multi-micro superlens, characterized in that It includes the following steps: An object to be measured, a telescope structure, a multi-eye micro-superlens, and an optical detector are sequentially arranged along the optical path direction; Obtain multi-eye detection images of the object to be measured at different set positions as a calibration image set. The multi-eye detection images include multiple imaging images collected by the optical detector after the object to be measured at the current position is imaged by each sub-eye of the multi-eye micro-superlens respectively; Select a pair of sub-eyes of the multi-eye micro-superlens to form a binocular ranging structure; Based on the calibration image set, fit the functional relationship between the distance from the object to be measured to the telescope structure and the imaging parallax value of the binocular ranging structure; Collect multi-eye detection images of the object to be measured at an unknown position, analyze to obtain the imaging parallax value of the binocular ranging structure, and substitute it into the functional relationship to calculate the predicted value of the distance from the current position of the object to be measured to the telescope structure.

2. The method according to claim 1, wherein The fitting of the functional relationship between the distance from the object to be measured to the telescope structure and the imaging parallax value of the first pair of sub-eyes based on the calibration image set includes: Obtain the baseline distance of the first pair of sub-eyes; Establish a mathematical model between the distance from the object to be measured to the telescope structure, the baseline distance of the first pair of sub-eyes, and its imaging parallax value; Based on the multi-eye detection images of the object to be measured at each set position, calculate the imaging parallax value of the first pair of sub-eyes at the corresponding position, and then use an optimization algorithm to fit the optimal parameters of the mathematical model to obtain the functional relationship.

3. The method according to claim 2, characterized in that The mathematical model is expressed as where Depth is the distance from the object to be measured to the telescope structure, B is the baseline distance between a pair of selected sub-eyes, d is the imaging parallax value of the pair of selected sub-eyes, and p1, p2, and p3 are model parameters.

4. The method according to claim 1, wherein It also includes selecting several different pairs of sub-eyes to obtain multiple binocular ranging structures, calculating the predicted distance values corresponding to each binocular ranging structure according to the multi-eye detection images, and then calculating the distance measurement value as the average value of all the predicted distance values.

5. The method according to claim 1, wherein The optical detector simultaneously collects imaging images of all sub-eyes of the multi-eye micro-superlens.

6. The method according to claim 1, characterized in that The telescope structure is installed on the multi-eye micro-superlens so that all sub-eyes of the multi-eye micro-superlens share the same optical path with the telescope structure.

7. The method according to claim 1, characterized in that, The telescope structure includes an objective lens and an eyepiece. After the object to be measured is imaged by the objective lens and the eyepiece of the telescope structure in sequence, it is imaged on the optical detector by each sub-eye of the multi-eye micro-superlens respectively.

Images