A three-dimensional measurement method for water-fire bent plates based on binocular vision

Abstract

The present invention discloses a three-dimensional measurement method for water and fire bent plates based on binocular vision. In this solution, a method of using an unmanned aerial vehicle equipped with a binocular camera is adopted to increase the flexibility of shooting, reduce the restriction on the size of the bent plate, and reduce the cost at the same time; secondly, a Census transformation method based on Gaussian weights can improve the matching accuracy in binocular matching, and at the same time is not sensitive to the overall brightness change, so even if the brightness of the left and right images is inconsistent, a better matching effect can be obtained; finally, on the basis of the traditional Canny algorithm, two directions of 45° and 135° are added, that is, the overall four directions are calculated, so as to highlight the edges with weaker gradient differences.

Description

Technical Field

[0001] The invention relates to a three-dimensional measurement technology for a water-fire bent plate, and in particular to a three-dimensional measurement method for a water-fire bent plate based on binocular vision. Background Art

[0002] The current method of three-dimensional measurement of water-fire bending plates, such as the invention patent "A method and system for measuring the angular deformation of water-fire bending plates" with publication number CN104359415B, usually uses a fixed-size device, or a frame, to place the water-fire bending plate in it and measure it using a camera that moves in a straight line. However, this method has great restrictions on the measurement object and cannot be moved at any time, which has great limitations. During the measurement process, the traditional checkerboard calibration method will extract some erroneous and redundant points, which brings great trouble to the extraction of the camera's internal and external parameters.

[0003] When performing calculations, the traditional Census transformation range is a square area of ​​pixels. Since the image of a certain area of ​​an object in space in a binocular photo is not exactly the same, the square neighborhood of the corresponding pixel will not be exactly the same. These problems will lead to inaccurate Census matching costs.

[0004] Similarly, the traditional Canny algorithm only considers the gradient difference in the x and y directions, which weakens the edge and increases the difficulty of later point cloud stitching. The Ransac algorithm takes a long time, has many registration errors, is inefficient, has a high time cost, and cannot meet the accuracy requirements. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a three-dimensional measurement method for water-fire bent plates based on binocular vision with high measurement accuracy, small error and low labor cost.

[0006] Technical solution: The present invention discloses a three-dimensional measurement method for water-fire bent plates based on binocular vision, comprising the following steps:

[0007] (1) Use a drone equipped with a binocular camera to shoot, calibrate the camera and correct its distortion to obtain the camera's intrinsic and extrinsic parameters;

[0008] (2) After epipolar correction of the photo, find the matching pixel points of all pixels in the left photo in the right photo. After obtaining the disparity map, perform cost matching calculation and edge detection using the Census matching cost and Canny algorithm respectively. When performing the Canny algorithm, add the calculation gradients in the directions of 45° and 135°.

[0009] (3) Based on the Gaussian weighted Census transformation method, point cloud coordinate calculation, point cloud denoising, point cloud stitching and surface three-dimensional reconstruction are performed to finally complete the three-dimensional reconstruction of the bent plate.

[0010] In the above technical solution, a drone equipped with a binocular camera is used to increase the flexibility of shooting, reduce the restrictions on the size of the curved plate, and reduce the cost; secondly, the Census transformation method based on Gaussian weights can improve the matching accuracy in binocular matching, and is not sensitive to the overall brightness changes. Therefore, even if the brightness of the left and right images is inconsistent, a good matching effect can be obtained; finally, on the basis of the traditional Canny algorithm, two directions of 45° and 135° are added, that is, the overall four directions are calculated to highlight the edges with weaker gradient differences.

[0011] Preferably, the camera is calibrated and distortion corrected using Zhang's calibration method.

[0012] Preferably, the specific process of camera calibration and distortion correction is to first prepare calibration images, extract corner point information for each calibration image, then further extract sub-pixel corner point information for each calibration image, and then draw the found inner corner points on the chessboard calibration map to calibrate the camera, and then evaluate the calibration results, check the calibration effect, and use the calibration results to correct the chessboard map.

[0013] Preferably, when performing the Canny algorithm, Otsu double threshold segmentation is added to perform edge detection to reduce redundant lines on the edge, which can also reduce the error of edge detection.

[0014] Preferably, the Otsu double threshold algorithm obtains the threshold g by the formula: g = ω0 (μ0-μ) 2 +ω1(μ1-μ) 2 +ω w (μ2-μ) 2 , where g is the maximum value, two thresholds can be obtained, and then the image after threshold segmentation is subjected to mean sharpening to obtain an edge image.

[0015] Preferably, the process of the Canny algorithm is as follows: input the original image, flatten the image through adaptive median filtering, and then calculate the gradient G in four directions of x, y, 45° and 135°. The formula is:

[0016] Preferably, in step (3), the specific steps of point cloud coordinate calculation, point cloud denoising, point cloud stitching and surface three-dimensional reconstruction are: first, a dense point cloud of the target surface is obtained by calculating the disparity map and the parameters of the photographic system, then the point cloud is denoised by the disparity map, and the denoised point cloud is stitched.

[0017] Preferably, the point cloud stitching is based on the point cloud stitching method of SAC_IA coarse alignment and NDT+ICP fine alignment, and KD tree is added to ICP. This method can retain the local characteristics of the bent plate and increase accuracy. KD_tree can speed up the search, reduce the running time, and improve work efficiency.

[0018] Beneficial effects: Compared with the prior art, the present invention has the following advantages: 1. The use of binocular vision reduces the restriction on the size of the bent plate; 2. The Census transformation method based on Gaussian weights can improve the matching accuracy in binocular matching; 3. On the basis of the Canny algorithm, the two directions of 45° and 135° are added, so that the edges with weaker gradient differences can be highlighted, thereby improving the measurement accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the weighted Census transformation template in the present invention;

[0020] Figure 2 This is a specific flow chart of step (2) in the present invention. DETAILED DESCRIPTION

[0021] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0022] The three-dimensional measurement method of water-fire bent plate based on binocular vision comprises the following steps:

[0023] (1) First, a drone equipped with a binocular camera is used to take photos. At the same time, the camera is calibrated and distortion corrected according to Zhang's calibration method to obtain the camera's internal and external parameters. The specific process of the calibration and distortion correction is as follows: first, prepare calibration images, extract corner point information from each calibration image, then further extract sub-pixel corner point information from each calibration image, then draw the found inner corner points on the chessboard calibration map, perform camera calibration, and then evaluate the calibration results to check the calibration effect and use the calibration results to correct the chessboard map;

[0024] (2) Then, the binocular matching method is used to find the matching pixel points of all pixels in the left photo in the right photo after epipolar correction of the photos, so as to obtain the disparity map, and the Census matching cost and Canny algorithm are used to perform cost matching calculation and edge detection respectively;

[0025] The traditional Census transformation range is the square neighborhood of pixels. Since the image of a certain area of ​​an object in space is not exactly the same in the binocular photo, the square neighborhood of the corresponding pixel will not be exactly the same. This will lead to inaccurate Census matching cost. Therefore, the Census transformation method based on Gaussian weights is adopted to improve the matching accuracy in binocular matching. In addition, the receptive field is also larger when the same binary storage capacity is used. Figure 1 As shown, this figure is a weighted Census transformation template;

[0026] At the same time, when performing the Canny algorithm, the calculated gradients in the directions of 45° and 135° are added, and the otsu double threshold segmentation is added for edge detection to reduce redundant lines on the edge. The specific algorithm flow is as follows: input the original image, smooth the image through adaptive median filtering, and then calculate the gradients G in the four directions of x, y, 45° and 135°: Then, the Otsu double threshold segmentation is used to process the image after the non-maximum value to determine the optimal threshold g for segmentation: g = ω0 (μ0-μ) 2 +ω1(μ1-μ) 2 +ω w (μ2-μ) 2 , when g is the maximum value, two can be selected, and then the image after threshold segmentation is mean sharpened, and finally the edge image is obtained, such as Figure 2 As shown, this figure shows the image processing process in this step;

[0027] (3) The Gaussian weighted Census transformation method is used to calculate point cloud coordinates, denoise point clouds, stitch point clouds, and reconstruct surface three-dimensional (3D) surfaces. Finally, the three-dimensional reconstruction of the fully bent plate is performed. The specific process is as follows: first, a dense point cloud of the target surface is obtained by calculating the disparity map and the parameters of the photographic system. Then, the point cloud is denoised by the disparity map and the denoised point cloud is stitched. The point cloud stitching is based on the SAC_IA coarse registration and NDT+ICP fine registration point cloud stitching method. At the same time, the KD tree is added to the ICP.

Claims

1. A three-dimensional measurement method for water-fire bent plates based on binocular vision, characterized in that: The following steps are involved: (1) Use a drone equipped with a binocular camera to take photos, calibrate the camera and correct its distortion to obtain the camera’s intrinsic and extrinsic parameters; (2) After epipolar correction of the photo, find the matching pixel points of all pixels in the left photo in the right photo. After obtaining the disparity map, use the Census matching cost and Canny algorithm to perform cost matching calculation and edge detection respectively. When performing the Canny algorithm, add the calculation gradients in the directions of 45° and 135°. (3) Based on the Gaussian weighted Census transformation method, point cloud coordinate calculation, point cloud denoising, point cloud stitching and surface 3D reconstruction are performed to finally complete the 3D reconstruction of the bent plate. In step (2), when performing the Canny algorithm, Otsu double threshold segmentation is added to perform edge detection to reduce redundant lines on the edge. The formula for calculating the threshold g by the Otsu double threshold algorithm is: , where g is the maximum value, two thresholds can be obtained, and then the image after threshold segmentation is subjected to mean sharpening to obtain the edge image. The process of the Canny algorithm is as follows: input the original image, flatten the image through adaptive median filtering, and then calculate the gradient G in four directions of x, y, 45° and 135°. The formula is: .

2. The three-dimensional measurement method of water-fire bending plate based on binocular vision according to claim 1 is characterized in that: In step (1), the camera is calibrated and distortion corrected using Zhang's calibration method.

3. The three-dimensional measurement method of water-fire bending plate based on binocular vision according to claim 2 is characterized in that: The specific process of camera calibration and distortion correction is to first prepare calibration images, extract corner information for each calibration image, and then further extract sub-pixel corner information for each calibration image. Then, draw the found inner corner points on the chessboard calibration map to calibrate the camera. After that, evaluate the calibration results, check the calibration effect, and use the calibration results to correct the chessboard map.

4. The three-dimensional measurement method of water-fire bending plate based on binocular vision according to claim 1 is characterized in that: In step (3), the specific steps of point cloud coordinate calculation, point cloud denoising, point cloud stitching and surface three-dimensional reconstruction are as follows: first, a dense point cloud of the target surface is obtained by calculating the disparity map and the parameters of the photographic system, then the point cloud is denoised by the disparity map, and the denoised point cloud is stitched.

5. The three-dimensional measurement method of water-fire bending plate based on binocular vision according to claim 4 is characterized in that: The point cloud stitching is based on a point cloud stitching method of SAC_IA coarse registration and NDT+ICP fine registration, and a KD tree is added to ICP.

Images