Binocular vision distance measurement method based on baseline distance

Abstract

The present invention relates to the field of binocular computer vision measurement, and in particular to a binocular vision distance measurement method based on baseline distance. The method comprises the following steps: S1. calibrating a binocular camera; S2. using the binocular camera to acquire images; S3. using a target recognition and feature point matching algorithm to calculate the parallax of target points in the left view imaged by the left camera and the right view imaged by the right camera; S4. offsetting the right view imaged by the right camera to the right by a distance C; and S5. adjusting the baseline distance so that the image parallax is 0, and obtaining the distance to be measured based on the adjusted baseline distance. When using this method for distance measurement, the problem of divergence of measurement accuracy at both ends in existing parallax distance measurement methods can be effectively solved, thereby extending the effective measurement range and increasing the accuracy.

Description

Technical Field

[0001] The present invention relates to the field of binocular computer vision measurement, in particular to a binocular vision distance measurement method based on baseline distance. Background Art

[0002] With the development of vision technology, binocular stereo vision is increasingly being used in fields such as 3D reconstruction and assisted driving. The need to convert 2D information captured by cameras into 3D information with depth is growing stronger. Existing binocular stereo vision uses the parallax method, which measures distance based on a fixed baseline distance and the parallax variation of the binocular camera imaging. Because parallax and distance are inversely proportional, measurement accuracy diverges at the ends, leaving only the middle segment with high accuracy. Therefore, existing binocular stereo distance measurement using parallax methods is primarily focused on a relatively small range. Summary of the Invention

[0003] The present invention aims to overcome the above-mentioned defects of the prior art and proposes a binocular vision ranging method based on baseline distance. When measuring distance using this method, the problem of divergence of measurement accuracy at both ends of the existing parallax ranging method can be effectively solved, making the effective measurement range wider and the accuracy higher.

[0004] The technical solution of the present invention is: a binocular vision distance measurement method based on baseline distance, which includes the following steps:

[0005] S1. Binocular camera calibration;

[0006] S2. Image acquisition using a binocular camera:

[0007] The object to be tested is imaged in the left camera and the right camera respectively;

[0008] S3. Calculate the parallax of the target point in the left view of the left camera and the right view of the right camera using the target recognition and feature point matching algorithm;

[0009] The target recognition algorithm is used to identify the target to be measured in the left and right views of the binocular camera imaging, and the circumscribed rectangle of the target to be measured in the image is obtained to obtain the coordinates of the circumscribed rectangle of the target to be measured. The feature point matching algorithm is used in the rectangular area, and the horizontal coordinates of the successfully matched feature points are averaged as the horizontal coordinate of the target point. The horizontal coordinate value in the left view of the left camera imaging is set to X l , the horizontal coordinate value of the right view of the right camera is X r , then the parallax of the target point in the left and right views is D = X l -X r ;

[0010] S4. Offset the right view of the right camera image to the right by a distance C:

[0011] To ensure that the image resolution remains unchanged, add C columns of pixels with a pixel value of 0 to the left of the image matrix of the right view imaged by the right camera. At the same time, delete the C columns of pixels on the rightmost side of the image matrix of the right view imaged by the right camera. Then, perform an offset process on the original pixel positions of the right view imaged by the right camera, where D-10<C≤D, and C should be selected to ensure that the object under test is fully displayed in the right view after the offset.

[0012] S5. Adjust the baseline distance so that the image parallax is 0, and obtain the measured distance based on the adjusted baseline distance;

[0013] Using the left camera as a reference, the right camera is horizontally translated so that the horizontal coordinates of the target point in the right view imaged by the right camera are equal to the horizontal coordinates of the target point in the left view imaged by the left camera. At this point, the parallax of the target point in the left and right views is zero, and the adjusted binocular camera baseline distance T is measured.

[0014] At this time, the true disparity between the left view and the right view is C, the baseline distance of the binocular camera is T, and the measured distance Z, that is, the distance between the object to be measured and the camera projection center, is obtained by the following formula:

[0015]

[0016] Where f is the focal length of the camera, and 1 / dx is the scaling factor from pixel units to length units;

[0017] S6 changes the distance between the object to be measured and the camera, and then repeats step S5 to obtain the changed measured distance Z;

[0018] If the parallax cannot be adjusted to 0 within the translation adjustment range of the right camera, repeat steps S2, S3, S4 and S5 to re-determine the offset distance C and obtain the changed distance to be measured Z.

[0019] In the present invention, in the above step S1, the binocular camera is calibrated to obtain the internal parameters of the camera. The distortion parameters of the camera are obtained through camera calibration. Each frame of the image taken by the camera is corrected using the distortion parameters to correct the deformation of the image caused by lens distortion. The binocular cameras are aligned to the same observation plane through epipolar correction, so that the imaging pixel rows of the binocular cameras are aligned.

[0020] In the above step S1, the binocular vision ranging device includes a left-eye camera, a right-eye camera, a stepper motor I, a stepper motor II, and a guide rail and screw slide body. The left-eye camera and the right-eye camera are both arranged on the guide rail and screw slide body. A baffle is provided between the left-eye camera and the right-eye camera, and the baffle is fixed to the guide rail and screw slide body. The stepper motor I is fixed to one end of the guide rail and screw slide body, and the stepper motor II is fixed to the other end of the guide rail and screw slide body. The output shaft of the stepper motor I is connected to the screw rod I through a coupling, and the other end of the screw rod I is rotatably connected to the baffle. The right-eye camera is fixed on the guide rail slide I, and the guide rail slide I is sleeved on the screw rod I. The screw rod I and the guide rail slide I are threadedly connected. When the screw rod I rotates, the guide rail slide I and the right-eye camera reciprocate along the axial direction of the screw rod I;

[0021] The output shaft of stepper motor II is connected to screw rod II through a coupling. The other end of screw rod II is rotatably connected to the baffle. The left eye camera is fixed on guide rail slide II. Guide rail slide II is sleeved on screw rod II. Screw rod II and guide rail slide II are threadedly connected. When screw rod II rotates, guide rail slide II and the left eye camera reciprocate along the axial direction of screw rod II.

[0022] The guide rail, screw rod and slide frame body is connected to the triangular support frame below it.

[0023] In the above step S2,

[0024] When the computer controls the stepper motor II to rotate the screw rod II, the guide rail slide II and the left eye camera fixed on the guide rail slide II are driven to move. The left view of the left eye camera image moves horizontally, and the object to be measured in the left view is adjusted to the middle of the view.

[0025] In the above step S5,

[0026] With the left view as a reference, the computer controls the stepper motor 1 to rotate the screw rod 1, which in turn drives the guide rail slide 1 and the right camera fixed on the guide rail slide 1 to move. The right view imaged by the right camera moves horizontally until the horizontal coordinate value of the target point in the right view is consistent with the horizontal coordinate value of the target point in the left view. At this time, the parallax of the horizontal coordinates of the target points in the left and right views is zero.

[0027] For the convenience of calculation, in the above step S4, C=10n (n=1, 2, ...).

[0028] The beneficial effects of the present invention are:

[0029] (1) The binocular vision distance measurement method based on baseline distance proposed in this application proposes an operation mode of obtaining zero image parallax by offsetting the right view and adjusting the baseline distance. By fixing the parallax C and changing the baseline distance T, a linear relationship is formed between the distance to be measured Z and the baseline distance T.

[0030] (2) During the measurement process, the problem of small measurement range and divergent accuracy at both ends of the existing binocular stereo vision ranging is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of the three-dimensional structure of a binocular visual ranging device;

[0032] Figure 2 This is a schematic diagram of the top view of the guide rail, screw rod and slide frame in the binocular vision ranging device;

[0033] Figure 3 This is a schematic diagram of the working state of the binocular vision ranging device;

[0034] Figure 4 This is a measurement flow chart of the binocular vision distance measurement method with variable baseline distance;

[0035] Figure 5(a) is the original image matrix before the right view is offset;

[0036] Figure 5(b) is the new image matrix after the right view is shifted;

[0037] Figure 6(a) is the binocular camera image after correction;

[0038] FIG6( b ) is a schematic diagram of the structure after the right view is horizontally shifted to the right by a distance of C pixels;

[0039] FIG6( c ) is a schematic diagram showing a structure in which the horizontal coordinates of the target points in the left view and the right view are consistent after the baseline distance is adjusted;

[0040] In the figure: 1 computer; 2 stepper motor I; 3 screw rod I; 4 right eye camera; 5 guide rail slide I; 6 screw rod II; 7 left eye camera; 8 guide rail slide II; 9 stepper motor II; 10 guide rail screw rod slide frame; 11 coupling; 12 tripod support frame; 13 calibration plate. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0042] The following description sets forth specific details to facilitate a thorough understanding of the present invention. However, the present invention can be implemented in a variety of other ways than those described herein, and those skilled in the art will be able to make similar generalizations without departing from the scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0043] The binocular vision distance measurement method based on baseline distance described in the present invention includes the following specific steps.

[0044] The first step is to connect the binocular vision ranging device to the computer and calibrate the binocular camera.

[0045] like Figures 1 to 3 As shown, the binocular vision ranging device includes a left camera 7, a right camera 4, a guide rail and screw slide 10, a stepper motor I2, a stepper motor II9, a screw I3, a screw II6, a guide rail slide I5, and a guide rail slide II8. The left camera 7 and the right camera 4 are both mounted on the guide rail and screw slide 10, with a baffle fixed to the guide rail and screw slide 10 between them.

[0046] A stepper motor II9 and a stepper motor I2 are fixedly mounted on the left and right ends of the guide rail and screw slide body 10, respectively. The output shaft of stepper motor II9 is ​​connected to a screw II6 via a coupling 11. The other end of screw II6 is rotatably connected to the baffle. The output shaft of stepper motor I2 is connected to a screw I3 via a coupling. The other end of screw I3 is rotatably connected to the baffle. Screws II6 and I3 rotate in conjunction with the rotation of stepper motors II9 and I2.

[0047] The right eye camera 4 is fixedly set on the guide rail slide I5, and the guide rail slide I5 is sleeved on the outside of the screw rod I3, and the screw rod I3 and the guide rail slide I5 are threadedly connected. The left eye camera 7 is fixedly set on the guide rail slide II8, and the guide rail slide II8 is sleeved on the outside of the screw rod II6, and the screw rod II6 and the guide rail slide II8 are threadedly connected. The guide rail slide II8 and the guide rail slide I5 can move left and right with the rotation of the screw rod II6 and the screw rod I3. The left eye camera 7 and the right eye camera 4 are respectively installed on the guide rail slide II8 and the guide rail slide I5, so that the positions of the left eye camera 7 and the right eye camera 4 can be adjusted by the stepper motor II9 and the stepper motor I2, and then the baseline distance of the binocular camera is changed. The guide rail screw slide frame 10 is connected to the triangular support frame 12 below it;

[0048] The binocular camera consisting of the left camera 7 and the right camera 4, the stepper motor I2, and the stepper motor II9 are connected to the computer 1 respectively. The binocular camera is calibrated using the Zhang Zhengyou calibration method. The camera distortion parameters are obtained through camera calibration. Each frame of the image taken by the camera is corrected using the distortion parameters to correct the deformation of the image caused by lens distortion. The epipolar line correction is used to align the binocular cameras to the same observation plane, so that the imaging pixel rows of the binocular cameras are aligned.

[0049] The second step is to use the binocular camera to collect images.

[0050] The left camera 7 and the right camera 4 are used to capture images of the object 13. The left camera 7 and the right camera 4 each capture an image of the object 13. Computer 1 controls stepper motor II9, which drives guide rail slide II8 to move horizontally. The left camera 7, mounted on guide rail slide II8, moves accordingly. The image of the object 13 in the left view captured by the left camera 7 is moved to the center of the view as a measurement reference.

[0051] In this embodiment, as shown in FIG6( a ), since the camera labeling and epipolar constraint operations are performed in the first step, the corresponding points in the left view and the right view are horizontally aligned.

[0052] In the third step, the target recognition and feature point matching algorithm is used to calculate the parallax of the target points in the left view imaged by the left camera and the right view imaged by the right camera.

[0053] Since there is a baseline distance between the binocular cameras, the horizontal coordinate value X of a point P in the left view of the left camera is l The horizontal coordinate value X in the right view of the right camera image r Different, and X l Always better than X r big.

[0054] The target recognition algorithm is used to identify the target to be measured in the left and right views of the binocular camera imaging, and the coordinates of the bounding rectangle of the target to be measured are obtained by circumscribing the rectangle of the target to be measured in the image. In order to ensure the accuracy of the disparity value, this application adopts the feature point matching algorithm, and the horizontal coordinates of the successfully matched feature points in the rectangle are averaged as the horizontal coordinate of the target point. The horizontal coordinate value in the left view of the left camera imaging is set to X l , the horizontal coordinate value of the right view of the right camera is X r , then the parallax of the target point in the left and right views is X l -X r .

[0055] In this embodiment, an optical calibration plate with 8*11 corner points and a rectangular side length of 25 (mm) is used as the object to be measured 13. In order to ensure the accuracy of the disparity value, the horizontal coordinates of the target point are obtained by taking the average of multiple feature points. First, the corner point detection algorithm is used to detect the 8*11 corner points of the object to be measured 13 in the left view and the right view. Taking the left view as an example, the coordinates of the 8*11 corner points of the object to be measured 13 are averaged as the coordinates of the target point, and the horizontal coordinate of the target point is taken as the X coordinate. l Similarly, the horizontal coordinate of the target point in the right view can be taken as the X r , so that the target point disparity in the two views is D(px)=X l -X r .

[0056] In this embodiment, Xl =969.3(px), X r =763.9 (px), then D = X l -X r =969.3-763.9=205.4(px).

[0057] The fourth step is to offset the right view of the right camera image to the right by a distance C (px).

[0058] To ensure unchanged image resolution, add C columns of zero pixels to the left side of the image matrix of the right view captured by the right camera. At the same time, delete the C rightmost columns of pixels in the image matrix of the right view captured by the right camera. The original pixel positions of the right view captured by the right camera are offset, where D - 10 < C ≤ D. C is selected so that the object under test is fully displayed in the right view after the offset. In this embodiment, for ease of calculation, C = 10n (n = 1, 2, ...).

[0059] In this embodiment, the meaning of the rightward offset of the right view is explained by distance. As shown in Figures 5(a) and 5(b), the 8*8 image is offset to the right by 2 (px). In other words, two columns of pixels with a pixel value of 0 are added to the left side of the original image matrix, and the two rightmost columns of pixels of the original image are deleted. This not only maintains the image resolution but also offsets the original pixel positions of the image. For example, the pixel with a pixel value of 123 in the original image has a coordinate of (0,0), and after the offset, the coordinate becomes (2,0), and the horizontal coordinate changes from 0 to 0+2.

[0060] As can be seen above, the right view of the right camera is offset to the right by a distance C. That is, C columns of pixels with a pixel value of 0 are added to the left side of the original image, and C columns of pixels on the right side of the original image are deleted. In this way, the positions of the remaining pixels in the original image are shifted to the right by C.

[0061] In this embodiment, the left view imaged by the left camera is used as a reference, and the right view imaged by the right camera is offset to the right by 200 (px), as shown in Figure 6(b). Comparing the horizontal coordinates of the target point in the right view before the offset in Figure 6(a), the horizontal coordinates of the target point in the right view after the offset in Figure 6(b) are r It becomes 763.9+200=963.9 (px).

[0062] The fifth step is to adjust the baseline distance so that the image parallax is 0, and calculate the distance to be measured based on the measured baseline distance.

[0063] With the left camera 7 as the reference, the right camera 4 is translated horizontally. During the translation process, the horizontal coordinate of the target point in the right view imaged by the right camera changes in real time. When the horizontal coordinate of the target point in the right view imaged by the right camera is equal to the horizontal coordinate of the target point in the left view imaged by the left camera, the parallax of the target point in the left and right views is zero. The movement of the right camera 4 is stopped, and the adjusted binocular camera baseline distance T is measured. The actual parallax value of the left and right views is C.

[0064] In this embodiment, the left eye camera 7 is used as a reference, and the computer 1 controls the stepper motor 12 to drive the guide rail slide 15 and the right eye camera 4 fixed on the guide rail slide 15 to move horizontally to the horizontal coordinate value X of the target point in the right eye camera 4. r =X l As shown in Figure 6(c), after the right camera 4 is horizontally translated, the horizontal coordinate of the target point in the left view of the left camera imaging is X l =969.3 (px), the horizontal coordinate of the target point in the right view of the right camera is X r =969.3 (px), at this time the parallax value of the target point in the left and right views obtained through the image is zero.

[0065] Since the zero parallax of the target point in the image is achieved by offsetting the right view by C = 200 pixels, the actual parallax between the left and right views is now C = 200 (px). The distance between the left camera 7 and the right camera 4 is measured, and the adjusted baseline distance of the binocular camera is obtained as T = 113.2 mm.

[0066] Substituting the baseline distance T and the true parallax value C measured in the above steps into the following relationship, the distance to be measured Z can be obtained:

[0067]

[0068] Where f is the focal length of the camera, 1 / dx is the scaling factor from pixel units to length units, and f / dx represents the product of the focal length f and the scaling factor 1 / dx from pixel units to length units, which is an internal parameter of the camera. At the same time, the true parallax value C is also a known number, then Therefore, from the above formula, we can know that the distance to be measured Z is proportional to the baseline distance T.

[0069] In this embodiment, f / dx=2742.6, C=200 (px), and T=113.2 (mm). Substituting these into the equations, we can obtain that the distance Z between the object under test 13 and the projection center of the binocular camera is 1552.3 (mm).

[0070] Step 6: Change the distance between the object to be measured and the camera, and repeat step 5 to obtain the changed distance Z to be measured.

[0071] If the right camera 4 cannot be adjusted to a parallax of 0 within the translation adjustment range due to the length limitation of the guide rail, screw rod slide body 10, repeat the second, third, fourth and fifth steps, reselect the offset distance C, and obtain the changed distance Z to be measured.

[0072] Therefore, when using the binocular vision ranging method based on the baseline distance to measure distance, it is only necessary to determine the C value to simply and accurately obtain the distance to be measured according to the method of the present invention.

[0073] The above is a detailed introduction to the binocular visual ranging method based on baseline distance provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention. The above description of the disclosed embodiments enables professionals in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A binocular vision ranging method based on baseline distance, characterized in that: The following steps are involved: S1. Binocular camera calibration; S2. Image acquisition using a binocular camera: The object to be tested is imaged in the left camera and the right camera respectively; S3. Calculate the parallax of the target point in the left view of the left camera and the right view of the right camera using the target recognition and feature point matching algorithm; The target recognition algorithm is used to identify the target to be measured in the left and right views of the binocular camera imaging, and the circumscribed rectangle of the target to be measured in the image is obtained to obtain the coordinates of the circumscribed rectangle of the target to be measured. The feature point matching algorithm is used in the rectangular area, and the horizontal coordinates of the successfully matched feature points are averaged as the horizontal coordinate of the target point. The horizontal coordinate value in the left view of the left camera imaging is set to X l , the horizontal coordinate value of the right view of the right camera is X r , then the parallax of the target point in the left and right views is D = X l -X r ; S4. Offset the right view of the right camera image to the right by a distance C: In order to ensure that the image resolution remains unchanged, add C columns of pixels with a pixel value of 0 on the left side of the image matrix of the right view imaged by the right camera, and delete the C columns of pixels on the rightmost side of the image matrix of the right view imaged by the right camera. Then, perform an offset process on the original pixel positions of the right view imaged by the right camera, where D-10<C≤D, and C is selected so that the object under test is fully displayed in the right view after the offset. S5. Adjust the baseline distance so that the image parallax is 0, and obtain the measured distance based on the adjusted baseline distance; Using the left camera as a reference, the right camera is horizontally translated so that the horizontal coordinates of the target point in the right view imaged by the right camera are equal to the horizontal coordinates of the target point in the left view imaged by the left camera. At this point, the parallax of the target point in the left and right views is zero, and the adjusted binocular camera baseline distance T is measured. At this time, the true disparity between the left view and the right view is C, the baseline distance of the binocular camera is T, and the measured distance Z, that is, the distance between the object to be measured and the camera projection center, is obtained by the following formula: Where f is the focal length of the camera, and 1 / dx is the scaling factor from pixel units to length units; S6 changes the distance between the object to be measured and the camera, and then repeats step S5 to obtain the changed measured distance Z; If the parallax cannot be adjusted to 0 within the translation adjustment range of the right camera, repeat steps S2, S3, S4 and S5 to re-determine the offset distance C and obtain the changed distance to be measured Z.

2. The method according to claim 1, characterized in that In step S1, the binocular camera is calibrated to obtain the camera's internal parameters. The camera's distortion parameters are obtained through camera calibration. Each frame of the image taken by the camera is corrected using the distortion parameters to correct the deformation of the image caused by lens distortion. The binocular cameras are aligned to the same observation plane through epipolar correction, so that the binocular camera imaging pixel rows are aligned.

3. The method according to claim 1, characterized in that In step S1, the binocular vision ranging device includes a left-eye camera, a right-eye camera, a stepper motor I, a stepper motor II, and a guide rail and screw slide body. The left-eye camera and the right-eye camera are both arranged on the guide rail and screw slide body. A baffle is provided between the left-eye camera and the right-eye camera, and the baffle is fixed to the guide rail and screw slide body. The stepper motor I is fixed to one end of the guide rail and screw slide body, and the stepper motor II is fixed to the other end of the guide rail and screw slide body. The output shaft of the stepper motor I is connected to the screw rod I through a coupling, and the other end of the screw rod I is rotatably connected to the baffle. The right-eye camera is fixed on the guide rail slide I, and the guide rail slide I is sleeved on the screw rod I. The screw rod I and the guide rail slide I are threadedly connected. When the screw rod I rotates, the guide rail slide I and the right-eye camera reciprocate along the axial direction of the screw rod I; The output shaft of stepper motor II is connected to screw rod II through a coupling. The other end of screw rod II is rotatably connected to the baffle. The left eye camera is fixed on guide rail slide II. Guide rail slide II is sleeved on screw rod II. Screw rod II and guide rail slide II are threadedly connected. When screw rod II rotates, guide rail slide II and the left eye camera reciprocate along the axial direction of screw rod II. The guide rail, screw rod and slide frame body is connected to the triangular support frame below it.

4. The method according to claim 3, characterized in that In the above step S2, When the computer controls the stepper motor II to rotate the screw rod II, the guide rail slide II and the left eye camera fixed on the guide rail slide II are driven to move. The left view of the left eye camera image moves horizontally, and the object to be measured in the left view is adjusted to the middle of the view.

5. The method according to claim 3, characterized in that In the above step S5, With the left view as a reference, the computer controls the stepper motor 1 to rotate the screw rod 1, which in turn drives the guide rail slide 1 and the right camera fixed on the guide rail slide 1 to move. The right view imaged by the right camera moves horizontally until the horizontal coordinate value of the target point in the right view is consistent with the horizontal coordinate value of the target point in the left view. At this time, the parallax of the horizontal coordinates of the target points in the left and right views is zero.

6. The method according to claim 1, characterized in that In the above step S4, C=10n (n=1, 2, ...).

Images