A deep loosening disturbance surface extraction device based on binocular vision

Abstract

The application relates to a deep loosening disturbance surface extraction device based on binocular vision. The device adopts Zhang Zhengyou calibration method to obtain camera internal parameters, and camera calibration is completed by shooting multiple plane target images at different angles; the SGBM algorithm is used to calculate the depth information of the deep loosening disturbance surface image, an initial disparity map is generated by selecting the disparity of corresponding pixel points of left and right eye images, a global energy function is constructed, and the optimal disparity is solved by minimizing the energy function, so that accurate matching of the binocular image is realized. In the implementation process, the deep loosening disturbance soil after deep loosening is removed in an artificial digging manner, so that the deep loosening groove bottom line is exposed; the collecting device is moved above the groove bottom line, and the rack is kept horizontal by means of the level; the Intel RealSense depth camera D435i is slowly and smoothly moved along the track by the motor control, and the image collection of the whole one-way stroke is completed. The application can efficiently and accurately extract the deep loosening disturbance surface information and is suitable for detecting and evaluating the quality of agricultural deep loosening operation.

Description

Technical Field

[0001] This invention relates to the field of deep soil loosening technology, and more specifically, to a device for extracting deep loosening disturbance surfaces based on binocular vision. Background Technology

[0002] Deep tillage, as an important method of land conservation tillage, can deepen the topsoil and break up the plow pan. The shape of the disturbed surface is an important factor affecting crop growth and the overall deep tillage effect in deep tillage between crop rows and in general deep tillage. Currently, the main method for determining the area of ​​the deep tillage disturbed surface is to manually select several cross-sectional points on the bottom line of the deep tillage furrow after the deep tillage operation and dig trenches. Then, the shape of the bottom line of the deep tillage furrow is measured using the slab method or laser method.

[0003] However, the plate insertion method is inefficient and has poor accuracy, especially when a large amount of data needs to be collected, which wastes a lot of time, manpower and resources; while the laser method is greatly affected by the measurement accuracy when the light intensity is high, and the cost is high.

[0004] Binocular stereo vision is a method based on the principle of parallax and using imaging devices to obtain three-dimensional geometric information of the object being measured from different positions. Realsense depth cameras can quickly extract the distance between the camera and a point and output the coordinate values ​​of a series of captured points. They are also inexpensive and less affected by the environment.

[0005] Therefore, it is essential to design a method that applies binocular stereo vision technology to the extraction of the bottom line of deep loosening disturbed surface trenches. Summary of the Invention

[0006] To address the aforementioned issues, this method proposes a device for rapidly extracting deep loose perturbation surfaces using a binocular vision camera.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A device for extracting deep loose perturbation surfaces based on binocular vision mainly includes a frame, a slide rail, a Realsense D534i depth camera, a camera platform, and a level.

[0009] The slide rail is connected to the binocular vision camera via the camera platform, enabling the camera to move smoothly in a straight line. This avoids the impact of dropped frames from the binocular vision camera on the data results, thereby improving the accuracy of the obtained three-dimensional coordinates.

[0010] The binocular vision camera is installed below the platform and is parallel to the ground when in operation, with the camera taking vertical shots of the soil directly below.

[0011] The level is installed above the frame so that the position of the whole machine can be adjusted during operation to make it parallel to the ground and avoid measurement errors caused by poor soil flatness.

[0012] In the above technical solution, the disturbance surface extraction method includes the following steps:

[0013] Step S1: Obtain camera intrinsic parameters using Zhang Zhengyou's calibration method. Take multiple images of a planar target, such as a checkerboard pattern, at different angles using the camera. Then, perform camera calibration (i.e., solve for the camera's intrinsic and extrinsic parameters) by calculating and analyzing the corner points of the checkerboard pattern.

[0014] Step S2: Calculate the depth of the deep sparse perturbation surface image using the SGBM algorithm, select the disparity of each pixel at the corresponding position in the left and right eye images, generate an initial disparity map, set a global energy function, minimize the energy function, and solve for the optimal disparity as a representation for binocular image matching.

[0015] Step S3: Manually remove the soil above the bottom line of the ditch.

[0016] Step S4: Move the present invention above the bottom line of the ditch where the soil has been dumped, and adjust the position according to the level to keep the frame level.

[0017] Step S5: Control the RealSense depth camera D534i to move slowly and smoothly along the track using the motor until the entire one-way journey is completed.

[0018] Specifically, in step S1, the calibration of the binocular vision camera includes the following steps:

[0019] Step S11 yields a mapping matrix (homography matrix) H for each image.

[0020] A two-dimensional point is represented by m = (u, v). T This means that a three-dimensional point can be represented by M = (X, Y, Z). T It is indicated that its augmented matrix is as well as The relationship between a 3D point and its projected image points is as follows:

[0021] In the formula, s is an arbitrary standard vector, A is the matrix intrinsic parameter, R (rotation matrix) and t (translation vector) are extrinsic parameters.

[0022] Internal Reference:

[0023] In the formula, (u0, v0) is the principal point of the camera in the image coordinate system, α and β are the scale factors of the u and v coordinate axes on the image, and γ represents the perpendicularity of the image coordinate axes.

[0024] Assuming the template plane lies on the plane Z=0 in the world coordinate system, then:

[0025]

[0026] Homogeneous coordinates on the calibration template (chessboard) plane and These are the homogeneous coordinates of the points projected onto the chessboard grid and corresponding to the points on the camera's image plane.

[0027] At this point, we can obtain a 3×3 matrix, as follows:

[0028] H=(h1, h2, h3)=λA(r1 r2 t)

[0029] Using the homography matrix, the constraints on the intrinsic parameter matrix A can be obtained as follows:

[0030]

[0031] Step S12 uses the constraints to linearly solve the intrinsic parameter matrix A.

[0032] Assume it exists:

[0033]

[0034] In the formula, B is a symmetric matrix. After obtaining B based on the principle of absolute quadric surfaces, the matrix B is inverted, and the intrinsic parameter matrix A is derived from it. The extrinsic parameters R and t are calculated using A and the homography matrix H, as shown in the following formula:

[0035]

[0036] Step S13 Maximum Likelihood Estimation

[0037] The parameters are optimized using the maximum likelihood criterion. Assuming there are n images and m calibration points on the template plane, the maximum likelihood estimate can be obtained by minimizing the following formula:

[0038]

[0039] In the formula, m ij R is the image point of the j-th point in the i-th image; i Let t be the rotation matrix of i images; i M is the translation vector of the i-th image; j Let be the spatial coordinates of the j-th point; the initial estimated values ​​are obtained using the results of the linear solution above, and the initial values ​​of the radial distortion coefficients k1 and k2 are 0.

[0040] Zhang Zhengyou's calibration method does not take into account tangential distortion, which he believes can be ignored.

[0041] Specifically, in step S2, calculating the image depth using the SGBM algorithm includes the following steps:

[0042] Step S21, depth calculation based on binocular images, works by calculating the pixel-level difference between the corresponding points of the same point in the left and right images, combined with parameters obtained from binocular camera calibration, to calculate the coordinate information of the 3D scene. Depth calculation formula:

[0043]

[0044] In the formula: Z is the depth distance, f is the camera focal length, B is the stereo camera baseline value, X R X T , respectively, are the x-coordinates of the target point in the left and right images after epipolar correction, and d is the disparity value.

[0045] Step S22: Calculate the energy function. Select the disparity of each pixel at corresponding positions in the left and right eye images to generate an initial disparity map. Set a global energy function to minimize the energy function and solve for the optimal disparity as the binocular image matching representation. The energy function is shown below:

[0046]

[0047] In the formula: D is the disparity map; N p Let C(p, D) be the adjacent pixels of p; p When the parallax is D p The cost of pixel p; P1 is the penalty coefficient of set 1, which is the set of pixels whose disparity value differs from p by 1 among the pixels adjacent to p; P2 is the penalty coefficient of set 2, which is the set of pixels whose disparity value differs from p by more than 1 among the pixels adjacent to p.

[0048] In the above technical solution, the Realsense depth camera moves in a straight line in the Y direction and continuously measures the distance from the camera to the bottom line of the trench (i.e., the Z coordinate value). After the measurement is completed, the depth data is directly saved to the computer.

[0049] In the above technical solution, the camera intrinsic parameters are obtained by using the Zhang Zhengyou calibration method. Based on the known position information between the four corner points of the target, the three-dimensional coordinates of the target in world coordinates are obtained by using the PnP algorithm and the camera intrinsic and extrinsic parameters (this device mainly uses the Z coordinates of each point).

[0050] In the above technical solution, the SGBM algorithm is used to calculate the depth of the deep loose perturbation surface image.

[0051] The present invention has the following benefits:

[0052] 1) Compared with the plate method for measuring the disturbance surface, the present invention reduces the required manpower and material resources, can extract the disturbance surface more efficiently, and the extracted disturbance surface has higher accuracy. The data can be directly stored in the computer, which greatly saves the time of subsequent data processing.

[0053] 2) Compared with the laser method for measuring disturbed surfaces, the present invention is not subject to environmental constraints and can still obtain accurate data under high light intensity conditions.

[0054] 3) This invention is small in size, easy to carry, and low in cost, and can be used on a large scale in actual agricultural production. Attached Figure Description

[0055] The advantages of these and / or other aspects of the present invention will become apparent and readily understood from the following description of preferred embodiments taken in conjunction with the accompanying drawings, wherein,

[0056] Figure 1 This is a three-dimensional structural diagram of a device for extracting deep loose perturbation surfaces based on binocular vision.

[0057] In the diagram: 1 is a level, 2 is a camera platform, 3 is a guide rail, 4 is a slider, 5 is a binocular vision camera, and 6 is a frame.

[0058] Figure 2 This is a flowchart of the image acquisition process for the Realsense depth camera.

[0059] Figure 3 The image is captured by the binocular vision camera as it moves along the track in step S4.

[0060] Figure 4 This is an image of the bottom line of the deep pine trench extracted by the present invention. Detailed Implementation

[0061] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings. In this specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation thereof.

[0062] See Figure 1 This paper illustrates the overall structure of a deep loose perturbation surface extraction device based on a binocular vision camera. The extraction method of the device includes the following steps:

[0063] Step S1: Obtain camera intrinsic parameters using Zhang Zhengyou's calibration method. Take multiple images of a planar target, such as a checkerboard pattern, at different angles using the camera. Then, perform camera calibration (i.e., solve for the camera's intrinsic and extrinsic parameters) by calculating and analyzing the corner points of the checkerboard pattern.

[0064] Step S2: Calculate the depth of the deep sparse perturbation surface image using the SGBM algorithm, select the disparity of each pixel at the corresponding position in the left and right eye images, generate an initial disparity map, set a global energy function, minimize the energy function, and solve for the optimal disparity as a representation for binocular image matching.

[0065] Step S3: Manually remove the soil above the bottom line of the ditch.

[0066] Step S4: Move the present invention above the bottom line of the trench where the soil has been dumped, and adjust the position according to the level to keep the frame horizontal.

[0067] Step S5: Control the RealSense depth camera D534i to move slowly and smoothly along the track using the motor until the entire one-way journey is completed.

[0068] The above description of embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall concept of the present invention and should not be construed as limiting the present invention.

Claims

1. A device for extracting deep loose perturbation surfaces based on binocular vision, comprising a level (1), a camera platform (2), a guide rail (3), a slider (4), a binocular vision camera (5), and a frame (6), the process of extracting deep loose perturbation surfaces is as follows: Step S1: Obtain camera intrinsic parameters using Zhang Zhengyou's calibration method. Take multiple images of the planar target from different angles using the camera, and then perform camera calibration by calculating and analyzing the corner points of the checkerboard pattern. Step S2: Calculate the depth of the deep plural perturbation surface image using the SGBM algorithm. Select the disparity of each pixel at corresponding positions in the left and right eye images to generate an initial disparity map. Set a global energy function, minimize the energy function, and solve for the optimal disparity as a representation for binocular image matching. Step S3: Using manual excavation, the disturbed soil from the deep loosening process is removed, exposing the bottom line of the deep loosening trench. Step S4: Move the device above the bottom line of the trench where the soil has been dumped, and adjust its position according to the level to keep the frame horizontal. Step S5: Control the RealSense depth camera D435i to move slowly and smoothly along the track using a motor until the entire one-way journey is completed.

2. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... The binocular vision camera is connected to the camera platform, ensuring that the camera is parallel to the platform and that the camera's shooting direction is perpendicular to the horizontal plane.

3. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... The calibration of the binocular vision camera in step S1 includes the following steps: Step S11 obtains a mapping matrix H for each image. Step S12 uses constraints to linearly solve the intrinsic parameter matrix A Step S13: Perform maximum likelihood estimation The device was calibrated using the Zhang Zhengyou calibration method, without considering tangential distortion.

4. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... Step S2, which uses the SGBM algorithm to calculate the image depth, includes the following steps: Step S21 is based on depth calculation using binocular images. Its basic principle is to calculate the 3D scene coordinate information by combining the pixel-level difference between the corresponding points of the same point in the left and right images with the parameters obtained from the binocular camera calibration. Step S22: Calculate the energy function. Select the disparity of each pixel at the corresponding position in the left and right eye images, generate an initial disparity map, set a global energy function, minimize the energy function, and solve for the optimal disparity as a representation of binocular image matching.

5. The deep loose perturbation surface extraction device based on binocular vision according to claim 1, characterized in that... In step S3, the disturbed soil is thrown out manually, without exerting any force on the undisturbed soil.

6. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... In step S4, it is assumed that the shape of the trench bottom line is uniform when the deep loosening conditions remain unchanged.

7. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... In step S4, a level is used to ensure the position of the entire structure, so as to avoid measurement errors caused by poor flatness of farmland soil.

8. The device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... In step S5, the slider only needs to complete a one-way stroke to extract the entire bottom line shape of the trench, and there is no need to return after reaching one end.

9. A device for extracting deep loose perturbation surfaces based on binocular vision according to claim 1, characterized in that... In step S5, the slider should move forward at a steady speed to avoid data point loss due to excessive speed.

Images