A weed positioning method based on binocular vision

Abstract

This invention provides a weed localization method based on binocular vision, belonging to the field of computer vision technology. It solves the technical problem of accurate and efficient weed localization in agriculture. The technical solution includes the following steps: S1, binocular camera calibration to obtain detailed camera parameters; S2, binocular camera stereo correction; S3, stereo matching; S4, obtaining the three-dimensional coordinates of the weed center point. The beneficial effects of this invention are: This invention utilizes binocular vision technology to construct a weed localization algorithm, achieving precise weed localization and providing a practical and accurate weeding solution for laser weeding robots.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for locating weeds based on binocular vision. Background Technology

[0002] With the development of agriculture, the growth and reproduction of weeds have seriously affected the yield and quality of crops. Previous weed management methods included manual weeding and the use of chemical herbicides; however, these methods require high labor costs and can pollute the agricultural environment. Therefore, developing an efficient and precise method for weed location and management has become a current research hotspot.

[0003] To address the aforementioned issues, the paper "Research on Monocular Vision Method for 3D Target Localization" (Computer Software and Computer Applications, 2022) proposed a target localization method based on monocular vision. This method compensates for the high manpower and material costs associated with traditional management methods. By detecting individual objects in 3D and capturing the scene from a single perspective, it locates weeds, providing an emerging technology for intelligent weed control. However, with the continuous development of artificial intelligence technology, monocular vision has encountered problems such as inaccurate localization and low precision in depth estimation, environmental perception, and perspective.

[0004] Therefore, in order to improve and compensate for the problems of monocular vision in depth estimation, viewpoint, and environmental perception, it is necessary to develop a weed localization method based on binocular vision. By using binocular vision to construct a weed localization algorithm, the accuracy and processing speed of weed localization can be improved, and precise weed localization can be achieved, providing a practical and accurate weeding solution for laser weeding robots. Summary of the Invention

[0005] The purpose of this invention is to provide a weed location method based on binocular vision, which solves the technical problem of accurately and efficiently locating weeds in the agricultural field, provides technical support for precision weeding of crops, and improves the production efficiency and crop quality of farmland.

[0006] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a weed localization method based on binocular vision, comprising the following steps:

[0007] S1. Binocular camera calibration: Obtain detailed camera parameters;

[0008] (1) Making a calibration plate

[0009] This invention selects a 9×7 chessboard as the calibration plate for camera calibration, with each chessboard square measuring 28mm×28mm.

[0010] (2) Acquire calibration images

[0011] Fix the binocular camera in place, continuously adjust the position and angle of the calibration plate and the binocular camera, acquire the images obtained by the corresponding left and right cameras, and save them.

[0012] (3) Individual calibration of left and right cameras

[0013] The images captured by the left and right cameras are imported into the camera calibration program in MATLAB to calibrate the camera parameters.

[0014] S2, stereo correction for binocular cameras;

[0015] Due to a series of uncontrollable factors, such as camera manufacturing errors and errors in the manual setup of the binocular system, there is a certain angle between the imaging planes of the left and right cameras. This results in misalignment of the row pixels in the two images captured by the binocular cameras, causing errors in subsequent weed location.

[0016] Stereo correction can effectively solve the problem of non-parallel and non-coplanar images captured by a camera. This invention uses the Bouguet algorithm for stereo correction, and the specific process is as follows:

[0017] Decompose the rotation matrix R obtained from camera calibration in step S1 into r l and r r These two matrices can make the optical central axes of the binocular camera parallel to each other, thus making the left and right imaging planes coplanar.

[0018]

[0019]

[0020] Based on this, in order to ensure that the camera's baseline and imaging plane are parallel, a transformation matrix R needs to be constructed using the translation matrix T obtained after camera calibration. rect Using this transformation matrix, the poles of two cameras can be transformed to infinity. This pole is the intersection of the camera baseline and the imaging plane. When the pole is at infinity, it is equivalent to achieving line alignment.

[0021]

[0022] Since the imaging plane must ultimately be parallel to the baseline,

[0023]

[0024] Where T = [-T, T, 0], e1 and e2 are orthogonal. The result can be calculated by taking the cross product of e1 along the principal optical axis (0, 0, 1).

[0025]

[0026] e3 only needs to be orthogonal to e1 and e2, therefore we have

[0027] e3 = e2 × e1

[0028] Rotation transformation matrix After the solution is obtained, the images from the left and right cameras can be obtained using this matrix.

[0029] Transforming a plane to the same plane is as follows:

[0030]

[0031] In the above formula, R is the rotation matrix, T is the translation matrix, and R rect Let e1, e2, and e3 be the transformation matrix, and e1, e2, and e3 be vectors.

[0032] S3, Stereo Matching;

[0033] Stereo matching, in a binocular stereo vision system, involves comparing images captured by two cameras (left and right) to determine the relationship between corresponding points. Specifically, it identifies which pixel in the right image corresponds to the center point of a weed in the left image, thus determining the weed's position in three-dimensional space. Its main purpose is to acquire three-dimensional information of the same object from different viewpoints, enabling applications such as depth perception and spatial reconstruction.

[0034] S4. Obtain the three-dimensional coordinates of the center point of the weeds;

[0035] The basic principle of binocular vision is to use two cameras, left and right, to simultaneously capture images of the same object. By comparing the parallax (i.e., the offset of pixels in the left and right images) between corresponding points, the position of the object in three-dimensional space is calculated. Based on the principle of binocular vision, this invention uses the parallax values ​​of the pixels obtained in step S3, combined with the camera's intrinsic and extrinsic parameters, to calculate the three-dimensional coordinates of the center point of the weed.

[0036] As a further optimization scheme of the weed localization method based on binocular vision provided by the present invention, in step S3, the present invention is based on the SGBM semi-global stereo matching algorithm. First, a disparity value is selected for each pixel to form an initial disparity map. Then, a global energy function is set, which is related to the disparity map. Finally, the problem of minimizing the energy function is solved to obtain the optimal disparity value for each pixel.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] (1) Finding the optimal disparity map based on the SGBM semi-global stereo matching algorithm

[0039] In binocular vision, depth information can be obtained by matching images from the left and right perspectives, thus enabling 3D reconstruction of a scene. A disparity map is an image representing depth information, reflecting the offset of each pixel in the left and right perspectives. This invention utilizes the SGBM semi-global stereo matching algorithm to calculate the minimum value of the global energy function, thereby obtaining the optimal right matching point for each left imaging point. The SGBM algorithm is a stereo matching algorithm based on global optimization, which has higher matching accuracy and robustness compared to traditional local window-based algorithms. This invention, based on the SGBM semi-global stereo matching algorithm, finds the minimum value of the global energy function, thereby obtaining the optimal right matching point for the left imaging point and acquiring the optimal disparity map.

[0040] (2) Stereo correction of binocular cameras based on Bouguet algorithm

[0041] In stereo matching with binocular cameras, stereo correction is necessary because the images from the two viewpoints do not completely overlap, ensuring that the two images are parallel and coplanar. During matching, only a one-dimensional search is needed within the same row of pixels, reducing computational complexity. Simultaneously, stereo correction improves matching accuracy and stability. This invention employs the Bouguet algorithm for stereo correction of binocular cameras. The Bouguet algorithm is a binocular correction algorithm based on Zhang Zhengyou's calibration method. By calibrating the images from the left and right cameras, it obtains the camera's intrinsic and extrinsic parameters, thereby calculating the transformation matrix between the two cameras and achieving image correction. Compared to other algorithms, the Bouguet algorithm has advantages such as simple computation and high accuracy. This invention uses the Bouguet algorithm for stereo correction of binocular cameras, ensuring that the images are parallel and coplanar, so that stereo matching only requires a one-dimensional search within the same row of pixels, reducing computational complexity.

[0042] (3) This invention utilizes binocular vision technology to construct a weed positioning algorithm, thereby achieving precise positioning of weeds and providing a practical and accurate weeding solution for laser weeding robots.

[0043] ① Improved positioning accuracy: Binocular vision technology can observe the target from two perspectives and achieve more accurate three-dimensional positioning by calculating parallax, which can improve positioning accuracy compared to monocular vision technology.

[0044] ② Enhanced anti-interference capability: In weedy location scenarios, complex environments and large changes in lighting can easily lead to a decline in image quality, affecting the positioning effect of monocular vision systems. Binocular vision systems, on the other hand, can observe the target from two perspectives, enhancing the system's anti-interference capability. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0046] Figure 1 This is a schematic diagram of a calibration plate for a binocular vision-based weed location method provided by the present invention.

[0047] Figure 2 A schematic diagram of a binocular camera in a binocular vision-based weed location method provided by the present invention.

[0048] Figure 3 The weed location result is provided in the weed location method based on binocular vision provided by the present invention.

[0049] Figure 4 The flowchart of the weed location method based on binocular vision provided by the present invention is shown.

[0050] Figure 5 This is a schematic diagram of weed positioning in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] Example

[0053] See Figures 1 to 5 This embodiment provides a weed localization method based on binocular vision, including the following specific steps:

[0054] S1. Binocular camera calibration: Obtain detailed camera parameters;

[0055] (1) Making a calibration plate

[0056] The calibration board used for camera calibration refers to a chessboard image with a specific size. In this invention, a 9×7 chessboard is selected, with each chessboard square measuring 28mm×28mm.

[0057] (2) Acquire calibration images

[0058] Fix the binocular camera in place, continuously adjust the position and angle of the calibration plate and the binocular camera, acquire the images obtained by the corresponding left and right cameras, and save them;

[0059] (3) Individual calibration of left and right cameras

[0060] The images captured by the left and right cameras are imported into the camera calibration program in MATLAB to calibrate the camera parameters.

[0061] S2, stereo correction for binocular cameras;

[0062] Due to a series of uncontrollable factors, such as camera manufacturing errors and errors in the manual setup of the binocular system, there is a certain angle between the imaging planes of the left and right cameras. This results in misalignment of the row pixels in the two images captured by the binocular cameras, causing errors in subsequent weed location.

[0063] Stereo correction can effectively solve the problem of non-parallel and non-coplanar images captured by a camera. This invention uses the Bouguet algorithm for stereo correction, and the specific process is as follows:

[0064] Decompose the rotation matrix R obtained from camera calibration in step S1 into r l and r r These two matrices can make the optical central axes of the binocular camera parallel to each other, thus making the left and right imaging planes coplanar.

[0065]

[0066]

[0067] Based on this, in order to ensure that the camera's baseline and imaging plane are parallel, a transformation matrix R needs to be constructed using the translation matrix T obtained after camera calibration. rect Using this transformation matrix, the poles of two cameras can be transformed to infinity. This pole is the intersection of the camera baseline and the imaging plane; when the pole is at infinity, it is equivalent to achieving line alignment.

[0068]

[0069] Since the imaging plane must ultimately be parallel to the baseline,

[0070]

[0071] Where T = [-T, T, 0], e1 and e2 are orthogonal. The result can be calculated by taking the cross product of e1 along the principal optical axis (0, 0, 1).

[0072]

[0073] e3 only needs to be orthogonal to e1 and e2, therefore we have

[0074] e3 = e2 × e1

[0075] Rotation transformation matrix After the solution is obtained, the imaging planes of the left and right cameras can be transformed to the same plane using this matrix, as shown in the following equation:

[0076]

[0077] In the above formula, R is the rotation matrix, T is the translation matrix, and R rect Let e1, e2, and e3 be the transformation matrix, and e1, e2, and e3 be vectors.

[0078] S3. Stereo matching: The SGBM semi-global stereo matching algorithm is used to solve for the optimal disparity value of each pixel.

[0079] S4. Using the disparity values ​​of the pixels obtained in step S3, and in combination with the camera's intrinsic and extrinsic parameters, calculate the three-dimensional coordinates of the center point of the weeds.

[0080] by Figure 5 For example, first perform step S1 to obtain detailed parameters of the stereo camera, including...

[0081]

[0082]

[0083] Translation matrix [-123.0086 0.5029 -2.3001]

[0084]

[0085] Then, stereo correction of the binocular camera is performed based on the Bouguet algorithm; then, the minimum value of the global energy function is calculated using the SGBM semi-global stereo matching algorithm, thereby obtaining the optimal right matching point for each left imaging point; finally, the three-dimensional coordinates of the weed center point are calculated using the camera parameters and the disparity value of the weed center point.

[0086] The method described above combines stereo correction from a binocular camera with SGBM stereo matching algorithms, as well as the calculation of camera parameters and disparity values, thereby providing relatively accurate and reliable 3D coordinates of the weed center point. This is highly valuable for scenarios requiring weed analysis, target localization, or other related applications.

[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for locating weeds based on binocular vision, characterized in that, Includes the following steps: S1. Binocular camera calibration: Obtain detailed camera parameters; S2, stereo correction for binocular cameras; In step S2, the specific process of stereo correction based on the Bouguet algorithm is as follows: Decompose the rotation matrix R obtained from camera calibration in step S1 into... and These two matrices make the optical central axes of the binocular camera parallel to each other, and make the left and right imaging planes coplanar. ； ； Based on this, in order to ensure that the camera's baseline and imaging plane are parallel, a transformation matrix is ​​constructed using the translation matrix T obtained after camera calibration. The transformation matrix is ​​used to transform the poles of the two cameras to infinity. This pole is the intersection of the camera baseline and the imaging plane. When the pole is at infinity, it is equivalent to achieving row alignment. ； Since the imaging plane must ultimately be parallel to the baseline, ； in, , and Orthogonal is sufficient, through the principal optical axis direction and Perform the cross product to calculate... ； As long as with As long as it is orthogonal, then we have ； Rotation transformation matrix After the solution is obtained, the imaging planes of the left and right cameras can be transformed to the same plane using this matrix, as shown in the following equation: ； In the above formula, Let be a rotation matrix. It is a translation matrix. Let be the transformation matrix. It is a vector; S3, Stereo Matching; S4. Obtain the three-dimensional coordinates of the center point of the weeds.

2. The weed localization method based on binocular vision according to claim 1, characterized in that, In step S1, the binocular camera is calibrated, and the detailed parameters of the camera are obtained. The specific process is as follows: (1) Making a calibration plate The calibration plate used in camera calibration refers to a plate or panel with geometric structure and feature points. Camera calibration plates include checkerboard calibration plates and circular calibration plates. (2) Acquire calibration images Fix the binocular camera in place, continuously adjust the position and angle of the calibration plate and the binocular camera, acquire the images obtained by the corresponding left and right cameras, and save them; (3) Individual calibration of left and right cameras The images captured by the left and right cameras are imported into the camera calibration program in MATLAB to calibrate the camera parameters.

3. The weed localization method based on binocular vision according to claim 1, characterized in that, In step S2, due to a series of uncontrollable factors such as camera manufacturing errors and errors in the manual construction of the binocular system, the imaging planes of the left and right cameras have a certain angle, resulting in misalignment between the row pixels of the left and right images captured by the binocular cameras, which brings errors to the subsequent weed positioning; stereo correction is used to solve the problem of the images captured by the camera not being parallel and coplanar, and stereo correction is performed based on the Bouguet algorithm.

4. The weed localization method based on binocular vision according to claim 1, characterized in that, In step S3, stereo matching is performed in a binocular stereo vision system by comparing images captured by the left and right cameras to find the relationship between corresponding points, thereby obtaining the three-dimensional information of the same object from different perspectives and realizing depth perception and spatial reconstruction.

5. The weed localization method based on binocular vision according to claim 1, characterized in that, In step S3, based on the SGBM semi-global stereo matching algorithm, firstly, a disparity value is selected for each pixel to form an initial disparity map. Then, a global energy function is set, which is related to the disparity map. Finally, the problem of minimizing the energy function is solved to obtain the optimal disparity value for each pixel.

6. The weed location method based on binocular vision according to claim 1, characterized in that, In step S4, the specific process of obtaining the three-dimensional coordinates of the weed center point is as follows: The basic principle of binocular vision is to use two cameras, left and right, to simultaneously capture images of the same object. By comparing the parallax between corresponding points in the two images, the position of the object in three-dimensional space is calculated. Based on the principle of binocular vision, the three-dimensional coordinates of the center point of the weeds are calculated by using the parallax value of the pixel obtained in step S3 and combining it with the camera's intrinsic and extrinsic parameters.

Images