A stereo matching method based on laser stripe lines

Abstract

The application discloses a kind of stereomatching methods based on laser stripe line, including steps 1: left laser stripe image and right laser stripe image are acquired by left and right camera acquisition;Step 2: respectively to left laser stripe image and right laser stripe image are pretreated;Step 3: the laser stripe center line is solved using improved Steger algorithm;Step 4: the center line point of left laser stripe image is selected in turn, and line point in right laser stripe image is matched;Step 5: the disparity threshold of matching point is calculated.The present application implementation can realize the stereomatching of subpixel coordinate point, precision value is higher, is conducive to the recovery three-dimensional point cloud.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, in particular to a kind of stereomatching method based on laser stripe line. BACKGROUND

[0002] Binocular stereomatching algorithm is one of the most important processes in realizing three-dimensional reconstruction, when the corresponding relationship between three-dimensional space and image is determined, the disparity needs to be calculated, and the corresponding relationship of the point in three-dimensional space on left and right images must be known, which is the purpose of stereomatching. Through stereomatching technology, the corresponding relationship of the points in left and right images can be determined, so as to obtain the disparity and restore the three-dimensional information of the points.

[0003] Stereomatching technology is an important technology in binocular stereovision, and a large number of practical algorithms have been proposed, including many basic constraints. By applying these constraints to the matching algorithm, the matching difficulty can be effectively reduced and the speed of stereomatching can be improved. However, relative to the single constraint, error matching is easy to occur. On this basis, local, semi-global and global matching algorithms are proposed, which have higher matching accuracy than the previous single constraint, but the matching efficiency is lower and the speed is slower. Moreover, the above algorithms only process image pixels, and the accuracy cannot reach the sub-pixel level. Therefore, it is necessary to propose a high-precision and efficient stereomatching algorithm. SUMMARY

[0004] In order to overcome the deficiencies in the prior art, the present application provides a stereomatching method based on laser stripe line, which extracts the sub-pixel coordinate value of the laser stripe center line point based on Steger algorithm, and then combines single constraint condition and local cost matching. The matched points reach the sub-pixel level.

[0005] In order to achieve the above application purposes and solve the technical problems, the technical solutions adopted are as follows:

[0006] A stereomatching method based on laser stripe line, comprising the following steps:

[0007] Step 1: acquiring left and right laser stripe images collected by left and right cameras;

[0008] Step 2: pre-processing the left and right laser stripe images respectively;

[0009] Step 3: using improved Steger algorithm to obtain laser stripe center line;

[0010] Step 4: sequentially selecting the center line points of the left laser stripe image and matching them with the line points in the right laser stripe image;

[0011] Step 5: Calculate the disparity threshold of the matching points.

[0012] Further, the step 2 comprises: performing global thresholding processing on the left and right laser stripe images to extract the laser stripe region, and the calculation method is as follows:

[0013]

[0014] Wherein, f(x, y) is the pixel value at the image coordinates (x, y), g(x, y) represents the thresholded pixel value, and T is the set threshold.

[0015] Further, the step 3 comprises: extracting the laser stripe center line of the preprocessed left and right laser stripe images by using Steger algorithm, and the specific steps are as follows:

[0016] Step 31: Obtain the r x , r y , r xx , r xy and r yy of each pixel point of the image, and the formula is as follows:

[0017]

[0018] Wherein, r x represents the first-order partial derivative of the image along the x direction, r y represents the first-order partial derivative of the image along the y direction, r xx represents the second-order partial derivative of the image along the x direction, r xy represents the second-order mixed partial derivative of the image along the x direction and then along the y direction, r yy represents the second-order partial derivative along the y direction, and G(x, y) is a two-dimensional Gaussian function, and g(x, y) is a one-dimensional Gaussian function.

[0019] Step 32: Calculate the eigenvalue and eigenvector by using Hessian matrix, wherein the eigenvector corresponding to the maximum eigenvalue of the Hessian matrix corresponds to the normal direction of the light bar, and is represented by n x and n y , and the Hessian matrix is represented as:

[0020]

[0021] Step 33: Take the point (x0, y0) as the standard point, and perform second-order Taylor expansion on the stripe cross-section gray distribution function to obtain the sub-pixel coordinates (P x , P y ) = (x0+tn x , y0+tn y ) of the light stripe center line, wherein the calculation formula of t is as follows:

[0022]

[0023] wherein n x and n y respectively represent that the eigenvectors corresponding to the maximum eigenvalues of the Hessian matrix correspond to the normal direction of the light bar.

[0024] Further, the Steger algorithm has a large amount of calculation of the Hessian matrix, wherein 5 two-dimensional Gaussian convolutions (r x , r y , r xx , r xy and r yy ) are performed at each point, resulting in low calculation efficiency and causing the real-time performance of the system to decrease, and therefore, the separability and symmetry of the Gaussian convolution can be utilized to equivalently decompose the two-dimensional Gaussian kernel into a one-dimensional Gaussian row convolution and a one-dimensional Gaussian column convolution, reducing the amount of calculation from 5n 2 times of multiplication and addition to 10n times of multiplication and addition.

[0025] Further, the step 4 comprises: sequentially selecting the center line points of the left laser stripe image and matching the line points in the right laser stripe image, and the specific method of matching the left and right laser points is as follows:

[0026] Step 41: find the pixel coordinate value where the sub-pixel coordinate is located, and extract a region of 1*n size taking the pixel as the center coordinate, wherein n is the width of the center line of the laser stripe, if n is even, n takes n+1, and if n is odd, n remains unchanged;

[0027] Step 42: taking the same longitudinal coordinate as an example, find the same region on the right laser stripe image, and the matching method is as follows: extract a region of 1*n size in the right laser stripe image, calculate the divisors of the pixel values of the corresponding pixel points, and calculate the average value of all divisor results, and the calculation formula is as follows:

[0028]

[0029] wherein Sl j represents the pixel value of the pixel point of the selected region of the left image, Sr j represents the pixel value of the pixel point of the selected region of the right image, wherein j takes (1-n), the center coordinate is the pixel coordinate where the selected sub-pixel coordinate is located, S j represents the pixel value divisor of the corresponding pixel point, and S ave is the average value of the pixel point divisor;

[0030] If the difference between each divisor and the average value is within 0.05, it is considered that the corresponding pixel value point is found, and the specific judgment formula is as follows:

[0031] S j -S ave <0.05

[0032] Step 43: If there is a laser stripe line point in the pixel value point, the matching is successful directly, if there is no laser stripe line point in the pixel value point, the nearest laser stripe points above and below the pixel value are taken respectively, and the corresponding laser stripe line point coordinates are calculated by using the following method, the specific method is as follows:

[0033] x L =(x0-x1)×(y-y1) / (y0-y1)+x1

[0034] Wherein, x L represents the sub-pixel coordinates of the laser point, x0 and x1 respectively represent the horizontal coordinate values of the upper and lower laser stripe points closest to the pixel value, y0 and y1 respectively represent the vertical coordinate values of the upper and lower laser stripe points closest to the pixel value, and y represents the vertical coordinate value of the sub-pixel coordinate line point.

[0035] Compared with the prior art, the above technical scheme has the following advantages and positive effects:

[0036] 1. The global thresholding processing is adopted in the application, a preliminary denoising image is obtained, and the sub-pixel coordinate values of the laser stripe center line are extracted after the Steger algorithm operation;

[0037] 2. The improved Steger algorithm is adopted in the application, and compared with the traditional Steger algorithm, the speed is improved;

[0038] 3. The polar line constraint and the cost matching are combined in the application, so that the matching accuracy is higher and the calculation amount is relatively small;

[0039] 4. The stereo matching algorithm in the application combines image denoising, improved Steger algorithm, polar line constraint and cost matching, so that the accuracy and efficiency of the matching result are greatly improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical scheme of the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0041] Figure 1 is a flowchart of a stereo matching method based on laser stripe line of the application. DETAILED DESCRIPTION

[0042] The technical solutions of the present application will be described clearly and completely in connection with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, but not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0043] As shown in the figure, the embodiment discloses a stereo matching method based on laser stripe line, comprising the following steps: Figure 1 Step 1: obtaining left and right camera acquisition of left laser stripe image and right laser stripe image;

[0044] Step 2: respectively pre-processing left laser stripe image and right laser stripe image;

[0045] Step 2: respectively pre-processing left laser stripe image and right laser stripe image;

[0046] Further, the step 2 comprises: performing global thresholding processing on left and right laser stripe images, extracting laser stripe region, and the calculation method is as follows:

[0047]

[0048] Wherein, f(x, y) is the pixel value at the image coordinate (x, y), g(x, y) represents the pixel value after thresholding, and T is the set threshold.

[0049] Step 3: using improved Steger algorithm to obtain laser stripe center line;

[0050] Further, the step 3 comprises: using Steger algorithm to extract laser stripe center line of pre-processed left and right laser stripe images, and the specific steps are as follows:

[0051] Step 31: obtaining r x , r y , r xx , r xy and r yy of each pixel point of the image, and the formula is as follows:

[0052]

[0053] Wherein, r x represents the first order partial derivative of the image along the x direction, r y represents the first order partial derivative of the image along the y direction, r xx represents the second order partial derivative of the image along the x direction, r xy represents the second order mixed partial derivative of the image along the x direction and then along the y direction, and r yyLet G(x, y) denote the second-order partial derivative along the y-direction, where G(x, y) is a two-dimensional Gaussian function and g(x, y) is a one-dimensional Gaussian function.

[0054] To reduce computational load, the separability of Gaussian convolution is utilized to transform the above five two-dimensional convolutions on image pixels into ten one-dimensional convolutions.

[0055] Step 32: Calculate the eigenvalues ​​and eigenvectors using the Hessian matrix. The eigenvector corresponding to the largest eigenvalue of the Hessian matrix corresponds to the normal direction of the light stripe. Let n... x and n y The Hessian matrix is ​​represented as:

[0056]

[0057] Step 33: Using point (x0, y0) as the standard point, perform a second-order Taylor expansion on the gray-level distribution function of the fringe cross section to obtain the sub-pixel coordinates (P) of the center line of the light fringe. x P y )=(x0+tn x y0+tn y ), where the formula for calculating t is as follows:

[0058]

[0059] Where, n x and n y These represent the eigenvectors corresponding to the largest eigenvalue of the Hessian matrix, which correspond to the normal directions of the light stripes.

[0060] Furthermore, the Steger algorithm, due to the large computational cost of the Hessian matrix, requires 5 operations (r) for each point. x r y r xx r xy and r yy Two-dimensional Gaussian convolution leads to low computational efficiency and degrades system real-time performance. Therefore, the separability and symmetry of Gaussian convolution can be utilized to decompose the two-dimensional Gaussian kernel into an equivalent Gaussian row convolution and a Gaussian column convolution, reducing the computational cost from 5n... 2 The number of multiplication and addition operations is reduced to 10n multiplication and addition operations.

[0061] Step 4: Select the center line points of the left laser stripe image in sequence and match them with the line points in the right laser stripe image;

[0062] Furthermore, step 4 includes: sequentially selecting the center line points of the left laser stripe image and matching them with the line points in the right laser stripe image. The specific method for matching the left and right laser points is as follows:

[0063] Step 41: find the pixel coordinate value where the sub-pixel coordinate is located, take the pixel as the center coordinate, and extract a 1*n size region, where n is the width of the laser stripe center line, if n is even, then n takes n+1, and if n is odd, it remains unchanged;

[0064] Step 42: find the same region on the right laser stripe image by the method of epipolar constraint, taking the same longitudinal coordinate as an example, the matching method is as follows: extract a 1*n size region in the right laser stripe image, calculate the divisors of the pixel values of the corresponding pixel points, and calculate the average value of all divisors, the calculation formula is as follows:

[0065]

[0066] Wherein, Sl j represents the pixel value of the pixel point selected in the left image, Sr j represents the pixel value of the pixel point selected in the right image, and j takes (1-n), the center coordinate is the pixel coordinate where the selected sub-pixel coordinate is located, S j represents the pixel value of the corresponding pixel point, S ave is the average value of the divisors of the corresponding pixel points.

[0067] If the difference between each divisor and the average value is within 0.05, it is considered that the corresponding pixel value point is found, and the specific judgment formula is as follows:

[0068] S j -S ave <0.05

[0069] Step 43: if there is a laser stripe line point in the pixel value point, the matching is successful directly, if there is no laser stripe line point in the pixel value point, take the nearest laser stripe point above and below the nearest laser stripe point, and then calculate the corresponding laser stripe line point coordinate by using the following method, the specific method is as follows:

[0070] x L =(x0-x1)×(y-y1) / (y0-y1)+x1

[0071] Wherein, x L represents the sub-pixel coordinate of the laser point to be solved, x0 and x1 represent the horizontal coordinate values of the upper and lower laser stripe points closest to the pixel value, respectively, y0 and y1 represent the vertical coordinate values of the upper and lower laser stripe points closest to the pixel value, respectively, and y represents the vertical coordinate value of the sub-pixel coordinate line point.

[0072] Step 5: calculate the matching point disparity threshold.

[0073] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any changes or replacements within the technical scope disclosed by the present application, which can be easily thought by those skilled in the art, should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A stereo matching method based on laser stripe lines, characterized in that, Includes the following steps: Step 1: Acquire the left and right laser stripe images captured by the left and right cameras; Step 2: Preprocess the left and right laser stripe images respectively; Step 3: Use the improved Steger algorithm to find the center line of the laser stripe; Step 3 includes: extracting the center lines of the laser stripes in the preprocessed left and right laser stripe images using the Steger algorithm. The specific steps are as follows: Step 31: Extract each pixel of the image , , , and The formula is as follows: in, This represents the first-order partial derivative of the image along the x-direction. This represents the first-order partial derivative of the image along the y-direction. This represents the second-order partial derivative of the image along the x-direction. This represents the second-order mixed partial derivative of the image obtained by first differentiating along the x-direction and then with respect to the y-direction. Let G(x,y) denote the second-order partial derivative along the y-direction, G(x,y) be a two-dimensional Gaussian function, g(x,y) be a one-dimensional Gaussian function, and f(x,y) be the pixel value at the image coordinates (x,y). Step 32: Calculate the eigenvalues ​​and eigenvectors using the Hessian matrix. The eigenvector corresponding to the largest eigenvalue of the Hessian matrix corresponds to the normal direction of the light stripe. and The Hessian matrix is ​​represented as: Step 33: Using points Using the standard point, a second-order Taylor expansion of the gray-level distribution function of the fringe cross section is performed to obtain the sub-pixel coordinates of the center line of the light fringe. The formula for calculating t is as follows: in, and These represent the eigenvectors corresponding to the largest eigenvalue of the Hessian matrix, which correspond to the normal directions of the light stripes. Step 4: Select the center line points of the left laser stripe image in sequence and match them with the line points in the right laser stripe image; Step 4 includes: sequentially selecting the center line points of the left laser stripe image and matching them with the line points in the right laser stripe image. The specific method for matching the left and right laser points is as follows: Step 41: Find the pixel coordinate value where the sub-pixel coordinate is located. Using this pixel as the center coordinate, extract a region of size 1*n in the left laser stripe image, where n is the width of the center line of the laser stripe. If n is even, then n+1 is taken; if it is odd, it remains unchanged. Step 42: Taking the same vertical coordinate as an example, find the same region on the right laser stripe image. The matching method is as follows: Extract a region of size 1*n in the right laser stripe image, calculate the divisor of the pixel value of the corresponding pixel, and calculate the average of all divisor results. The calculation formula is as follows: in, This represents the pixel value of the selected region in the left image. This represents the pixel value of the selected region in the right image, where j takes the value (1-n), and the center coordinates are the pixel coordinates of the selected sub-pixel. This represents the divisor of the pixel value at the corresponding pixel. This is the average of the divisors for the corresponding pixels; If the difference between each divisor and the average is within 0.05, then the corresponding pixel value is considered to have been found. The specific formula is as follows: Step 43: If a laser stripe point exists within the pixel value, the match is successful. If no laser stripe point exists within the pixel value, the nearest laser stripe points above and below the pixel value are taken, and the coordinates of the corresponding laser stripe points are calculated using the following method: in, This represents the sub-pixel coordinates of the laser point being sought. , These represent the x-coordinates of the laser stripe points above and below that are closest to the pixel value. , y represents the ordinate of the laser stripe point above and below that is closest to the pixel value, and y represents the ordinate of the sub-pixel coordinate line point; Step 5: Calculate the disparity threshold of the matching point.

2. The stereo matching method based on laser stripe lines according to claim 1, characterized in that, Step 2 includes: performing global thresholding on the left and right laser stripe images to extract the laser stripe regions. The calculation method is as follows: Where h(x, y) represents the thresholded pixel value, and T is the set threshold.

3. The stereo matching method based on laser stripe lines according to claim 1, characterized in that, The Steger algorithm, due to the large amount of computation required for the Hessian matrix, involves performing the operation 5 times for each point. , , , and Two-dimensional Gaussian convolution leads to low computational efficiency and degrades system real-time performance. Therefore, the separability and symmetry of Gaussian convolution can be utilized to decompose the two-dimensional Gaussian kernel into an equivalent Gaussian row convolution and a Gaussian column convolution, reducing the computational cost from 5n... 2 The number of multiplication and addition operations is reduced to 10n multiplication and addition operations.

Images