Method for estimating tobacco leaf picking point and picking direction

Abstract

The invention relates to the technical field of computer vision identification of smart agriculture, and discloses a method for estimating a tobacco leaf picking point and a picking direction, which is used for performing instance segmentation on tobacco leaves based on a YOLACTT + + algorithm, combining point cloud data processing and estimating the picking point and the picking direction of the tobacco leaves aiming at a non-shielding condition. According to the method for estimating the picking point and the picking direction of the tobacco leaves, RGB and depth images of the tobacco leaves are captured through an Intel RealSense D435i camera, the image quality is improved through a depth completion algorithm, and high-quality input is provided for a model; in a backbone network of the model, a deformable convolutional layer DCN is adopted, the DCN adopts flexible sampling points to replace fixed grid sampling in a traditional convolutional neural network (CNN), the accuracy of detection and segmentation of the model is remarkably improved, and experimental results show that the picking point and the picking direction can be well estimated, and the accuracy of detection and segmentation of the picking point and the picking direction is improved. The automation level of tobacco leaf picking is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision recognition technology for smart agriculture, and specifically to a method for estimating tobacco leaf picking points and picking directions. Background Art

[0002] Traditional tobacco leaf picking relies on manual labor, which is inefficient and costly. Considering the trend of population aging and the decline in the number of laborers, automated picking will be the future direction of tobacco leaf picking. The picking point and picking direction of tobacco leaves are estimated to be the core steps of tobacco leaf picking, and are also relatively difficult problems, especially in the case of occlusion. There is currently no complete solution. Summary of the invention

[0003] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides a method for estimating tobacco picking points and picking directions, and adopts the YOLACT++ target detection algorithm for instance segmentation. In the backbone network of the model, a deformable convolutional layer DCN is adopted. By adopting flexible sampling points instead of fixed grid sampling in traditional convolutional neural networks (CNN), the accuracy of the detection and segmentation models is significantly improved.

[0004] (II) Technical solution In order to achieve the above-mentioned instance segmentation using the YOLACT++ target detection algorithm, a deformable convolution layer DCN is used in the backbone network of the model. By using flexible sampling points instead of fixed grid sampling in traditional convolutional neural networks (CNN), the accuracy of the detection and segmentation models is significantly improved. The present invention provides the following technical solutions: A method for estimating tobacco leaf picking points and picking directions, comprising the following steps: Use the depth camera to obtain RGB and depth images, and preprocess the depth images using the depth completion algorithm; Obtain tobacco leaf images, calibrate the images, and obtain training data sets; Use YOLACT++ optimized with instance segmentation algorithm and point cloud data to identify and estimate picking points and picking directions in unobstructed situations.

[0005] Preferably, the method for acquiring and preprocessing the depth image is as follows: Use Python language and depth measurement principle to capture color images, depth images and infrared images of Intel RealSense D435i camera The camera is calibrated and the depth image is preprocessed using a depth completion algorithm.

[0006] Preferably, the depth measurement principle is as follows: The Intel RealSense D435i depth camera combines the classic binocular vision method and infrared light projection technology to achieve accurate depth measurement. According to the principle of binocular imaging, the depth information of a certain point can be obtained by knowing the parallax of that point.

[0007] Preferably, the method of the calibration technology is as follows: Use Kalibr Visual-Inertial Calibration Toolbox to calibrate multiple cameras and transform image and IMU data into a common reference coordinate system Calibrate the IMU to eliminate the influence of errors on parameters during measurement The IMU is jointly calibrated with the RGB camera, and the IMU is jointly calibrated with the stereo camera to fuse the image data and the IMU data and align them to a common reference coordinate system.

[0008] Preferably, the depth completion algorithm is as follows: The algorithm optimizes the depth image through steps such as depth inversion, kernel expansion, small hole closing and large hole filling to complete the missing depth values, which is suitable for the subsequent determination of picking points and directions.

[0009] Preferably, the tobacco leaf picking point and picking direction estimation method is as follows: Calibration and training of tobacco leaf dataset; The picking area is determined to match the tobacco leaves according to a fixed threshold, and the tobacco plants are detected using the trained instance segmentation network to obtain the instance mask of the picking area, and the pixel center point of the picking area is calculated as the picking point; Use coordinate transformation and statistical filtering to preprocess the point cloud data and convert the depth map into a point cloud map; The camera's attitude is obtained through the IMU, and the point cloud is rotated so that the height in the real world is used as the value of the z-axis; PCA is used to estimate the normal vector of the picking area, and the picking direction is determined by the normal vector of the point cloud cluster near the picking point.

[0010] Preferably, the instance segmentation algorithm is as follows: The instance segmentation algorithm used is the YOLACT++ algorithm, which improves the segmentation accuracy and speed in complex environments through an enhanced network structure and deformable convolutional layer DCN.

[0011] Preferably, the method for matching the picking areas with tobacco leaves is as follows: When a mature tobacco leaf that needs to be picked is photographed, it is necessary to calculate the ratio of the intersection area of ​​the tobacco leaf prediction frame and the prediction frame of each picking area to the picking area. If the ratio is greater than a set fixed threshold, it is determined that the picking area belongs to the tobacco leaf.

[0012] Preferably, the ratio expression is as follows:

[0013] Where: The intersection area of ​​the prediction box for tobacco leaves and picking area; Predict the area of ​​the box for the picking area.

[0014] Preferably, the point cloud processing technology is as follows: In order to reconstruct the exact position of the pixel value in the real world, a series of coordinate transformations must be performed to convert the depth map into a point cloud map, converting the points from the pixel coordinate system to the physical coordinate system, and then to the camera coordinate system; Perform statistical filtering on the scattered noise points in the data, set the statistical filter's adjacent points to 50 and the threshold to , downsampling technology is used to reduce the amount of calculation.

[0015] Preferably, the method of acquiring the camera posture and rotating the point cloud is as follows: Use IMU to obtain the attitude of the Intel RealSense D435i camera. The yaw angle, pitch angle, and roll angle of the camera can be obtained by integration. The rotation matrix and the coordinates of the midpoint of the point cloud are used for matrix dot multiplication.

[0016] Preferably, the expression of the rotation is as follows: Around The axis rotates counterclockwise The formula is as follows:

[0017] Around The axis rotates counterclockwise The formula is as follows:

[0018] Around The axis rotates counterclockwise The formula is as follows:

[0019] Preferably, the PCA principal component analysis method is as follows: After the picking point of a specific picking area is determined, a spherical area with a radius of r centered at the picking point is obtained to form a point cloud set for the PCA method. ; Find the mean of the coordinate system direction, decentralize the vectors in each direction, and form a matrix , find the matrix The covariance matrix of is expanded and the expression is obtained. The singular value decomposition algorithm is used to decompose the covariance matrix to obtain the expression; Project the vector into three-dimensional space and express it as radius, polar angle and azimuth in spherical coordinates using axis-angle notation. Use the vector in three-dimensional rectangular coordinates to obtain the polar angle. and azimuth , convert the normal vector obtained by PCA into a polar angle and azimuth As a picking direction; Preferably, the expression is as follows: Find the matrix The covariance matrix of is:

[0020] Expand the covariance matrix, where the matrix :

[0021] Apply the SVD (singular value decomposition) algorithm to decompose the covariance matrix:

[0022] Three matrices can be obtained: , and ,exist In this diagonal matrix, the elements on the diagonal , , are the eigenvalues ​​of the covariance matrix, which are arranged in descending order; In the matrix, the corresponding three column vectors The eigenvectors corresponding to these eigenvalues ​​are: The corresponding eigenvector is the normal vector at that point.

[0023] Preferably, the PCA obtains the eigenvector as follows: Local plane H, first projected to the eigenvector On, so that the variance is maximized; then project it onto the eigenvector On the left, the variance is the second largest, and then projected to the third eigenvector When going up, the eigenvector is the normal vector of the plane.

[0024] Preferably, the axis-angle representation is as follows: In the spherical coordinate system, is the polar angle, which is the angle between the vector and the positive half axis of the z-axis, satisfying:

[0025] is the azimuth, which is the vector The angle between the projection on the plane and the positive x-axis satisfies:

[0026] The polar angle can be obtained using a vector in a three-dimensional rectangular coordinate system and azimuth , the formula is as follows:

[0027]

[0028] .

[0029] (III) Beneficial effects Compared with the prior art, the present invention provides a method for estimating tobacco leaf picking points and picking directions, which has the following beneficial effects: The method for estimating the picking point and picking direction of tobacco leaves performs instance segmentation of tobacco leaves based on the YOLACT++ algorithm, and estimates the picking point and picking direction of tobacco leaves in non-occluded situations in combination with point cloud data processing. The system uses the Intel RealSense D435i camera to capture RGB and depth images of tobacco leaves, and improves the image quality through the depth completion algorithm to provide high-quality input for the model. In the backbone network of the model, a deformable convolutional layer DCN is used. DCN significantly improves the accuracy of the detection and segmentation model by using flexible sampling points instead of fixed grid sampling in traditional convolutional neural networks (CNN). Experimental results show that the present invention can better estimate the picking point and picking direction, and improve the automation level of tobacco leaf picking. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of the network structure of YOLACT of the present invention; Figure 2 This is a Protonet branch structure diagram of the present invention; Figure 3 This is the Prediction Head branch structure diagram of the present invention; Figure 4 This is a structural diagram of a DCN module in the backbone network of the present invention; Figure 5 It is the Mask Re-Scoring branch of the present invention; Figure 6 It is a schematic diagram of coordinate system conversion of the present invention; Figure 7 This is a schematic diagram of the camera posture of the present invention; Figure 8 A schematic diagram of obtaining characteristic vectors for PCA of the present invention; Fig. 9 It is a schematic diagram of the spherical coordinate system of the present invention; Fig.10 This is the segmentation result diagram of the example of the present invention; Fig.11 This is an example diagram of tobacco leaf picking points of the present invention; Fig.12 This is an example diagram of the tobacco leaf picking direction of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the embodiments of the present invention and the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1-10 , Step 1, use the depth camera to obtain RGB and depth images, and preprocess the depth images using the depth completion algorithm, as follows: S1.1. Use the Intel RealSense D435i camera to capture scene information, pass it to the internal depth processing unit, and use binocular vision technology to determine the depth value of each pixel in the image; S1.2, based on the concept that light travels in a straight line, the specific position of a point is determined, and the depth information of the point is obtained from the parallax of the point; S1.3. Use the Kalibr visual-inertial calibration toolbox to calibrate multiple cameras and convert the image and IMU data into a common reference coordinate system. S1.4. Calibrate the IMU to eliminate the influence of errors on parameters during the measurement process; S1.5, jointly calibrate the IMU with the RGB camera, and the IMU with the stereo camera to fuse the image data and the IMU data and align these data into a common reference coordinate system; S1.6, optimizing the depth image through depth inversion, kernel expansion, small hole closing and large hole filling to complete the missing depth value; Step 2: Obtain tobacco leaf images and calibrate the images to obtain training data sets: S2.1, taking pictures of tobacco leaves that are not obstructed; S2.2. Manually annotate the above-mentioned images using an annotation tool, select the bounding box of the leaf vein, main stem, picking point, and leaf using the annotation tool, and ensure that the bounding box fits closely with the target contour; Step 3: Use YOLACT++ optimized with instance segmentation algorithm and point cloud data to identify and estimate the picking point and picking direction in the unobstructed case.

[0033] In the process of implementing instance segmentation, YOLACT relies on two branches that work together. The architecture mainly consists of two parts: the Prediction Head branch is responsible for generating the category confidence, positioning regression parameters and mask coefficients of each anchor point; the Protonet branch is responsible for generating a set of prototype masks. Through the combination of these prototype masks and mask coefficients, a unique mask can be generated for each individual entity in the image. See the network structure diagram of YOLACT. Figure 1 .

[0034] The Protonet branch is composed of several convolutional layers. The model receives the P3 feature map as input and outputs a mask with a dimension of 138*138*k (where k=32). There are 32 prototype masks, each with a size of 138*138. See the Protonet branch structure diagram for details. Figure 2 .

[0035] The Prediction Head branch is optimized based on RetinaNet. It uses a shared convolutional network architecture to improve processing speed and achieve real-time image segmentation. This branch processes five feature maps from P3 to P7. Each feature map will first generate anchor points. Three anchor points are generated for each pixel, and the ratios are 1:1, 1:2, and 2:1. The basic side lengths of the anchor points of these five feature maps are 24, 48, 96, 192, and 384, respectively. These side lengths will be adjusted according to different ratios. See the Prediction Head branch structure diagram. Figure 3 .

[0036] After being processed by the Prediction Head branch network, a large number of anchor points will be generated. By applying position offsets on these anchor points, the position of the region of interest (RoI) can be determined. Since there may be overlaps between RoIs, the non-maximum suppression (NMS) algorithm is usually used for selection. YOLACT introduces a new screening algorithm called Fast-NMS, which effectively reduces the time required for screening without sacrificing accuracy.

[0037] YOLACT++ is an advanced version of the YOLACT algorithm, which uses deformable convolutional networks (DCN). DCN uses flexible sampling points instead of fixed grid sampling in traditional convolutional neural networks (CNN), which significantly improves the accuracy of detection and segmentation models. The YOLACT++ algorithm draws on the design of Deformable ConvNets v2 and improves the C3 to C5 parts of ResNet. It replaces the standard convolutions in these layers with deformable convolutions. In order to balance performance and computational efficiency, one standard convolution in every three convolutional layers is replaced by a deformable convolution. See the DCN module structure diagram in the backbone network. Figure 4 .

[0038] In the model structure of YOLACT++, a Mask Re-Scoring branch is added. This branch receives the mask without threshold processing as input and calculates the intersection over union (IoU) between the mask of each category and the true mask (GT-mask) to more accurately evaluate the quality of the mask. See the Mask Re-Scoring branch structure diagram Figure 5 .

[0039] S3.1. Set the training parameters such as training batch size, maximum number of training times, learning rate, etc., and use the training data set to train the model; S3.2, determine the picking area matching tobacco leaves according to a fixed threshold, use the trained instance segmentation network to detect the tobacco plants, obtain the instance mask of the picking area, and calculate the pixel center point of the picking area as the picking point; S3.3, pre-processing the point cloud data using coordinate transformation and statistical filtering to convert the depth map into a point cloud map; The depth map obtained by the Intel RealSense D435i depth camera assigns a value to each pixel. This value represents the distance from the corresponding point in the three-dimensional space to the camera. In order to reconstruct their exact position in the real world from these values, that is, the point cloud in the camera coordinate system, a series of coordinate transformations must be performed. The coordinate system parameter table is as follows:

[0040] To locate pixel coordinates in physical space, the pixel positions (i.e. rows and columns) of the depth map must be mapped to their distances in the real world. This mapping process can be described by the following formula:

[0041] Where: and Represents the actual size of each pixel in the physical coordinate system; and Represent the column and row coordinates in the original pixel coordinate system respectively.

[0042] Use the matrix to represent the above formula:

[0043] The specific conversion formula is as follows:

[0044] Where: Indicates the depth in the camera's field of view; and Represent the focal length of the camera in the x-axis and y-axis directions respectively; the optical center coordinates are set as .

[0045] Coordinate system transformation diagram Figure 6 .

[0046] S3.4, calibrate the IMU to eliminate the influence of errors on parameters during the measurement process; S3.5. Obtain the camera's posture through the IMU and rotate the point cloud so that the height in the real world is used as the value of the z-axis; The purpose of point cloud rotation is to use the height in the real world as the value of the z-axis. To rotate the point cloud, you first need to obtain the posture of the Intel RealSense D435i camera, which is obtained through the IMU. Each IMU frame is updated and processed asynchronously. The yaw angle, pitch angle, and roll angle of the camera can be obtained by integration.

[0047] Camera posture diagram Figure 7 .

[0048] S3.6, PCA is used to estimate the normal vector of the picking area, and the picking direction is determined by the normal vector of the point cloud cluster near the picking point; After determining the picking point of a specific picking area, a spherical area with a radius of r centered at the picking point is obtained. All points in the spherical area constitute a point cloud set for the PCA method, which is shown in the following formula:

[0049] Where: are the coordinate values ​​of the above point cloud set at the three coordinates of x, y, and z Take the mean over the coordinate system directions:

[0050] Decentralize the vectors in each direction to form a matrix ;

[0051] Find the matrix The covariance matrix of After decentralization, ;

[0052] By applying the SVD (singular value decomposition) algorithm to decompose the covariance matrix, as shown in formula (3-17), three matrices can be obtained: , and ,exist In this diagonal matrix, the elements on the diagonal , , are the eigenvalues ​​of the covariance matrix, which are arranged in descending order; In the matrix, the corresponding three column vectors The eigenvectors corresponding to these eigenvalues ​​are: The corresponding eigenvector is the normal vector at that point.

[0053]

[0054] See the schematic diagram of PCA obtaining eigenvectors for Figure 8 .

[0055] Axis-angle representation is a common method, which is expressed as radius, polar angle and azimuth in spherical coordinate system. The spherical coordinate system is shown in the figure below. In the spherical coordinate system, is the polar angle, which is the angle between the vector and the positive half axis of the z-axis, satisfying:

[0056] is the azimuth, which is the vector The angle between the projection on the plane and the positive x-axis satisfies: .

[0057] The schematic diagram of the spherical coordinate system is shown in Fig. 9 .

[0058] Step 4: Conduct an estimation experiment on the picking point and picking direction in the greenhouse when the picking area is not blocked, as follows: For the unobstructed picking area, we conducted 30 experiments. Through manual observation, there were 146 unobstructed leaves in the 30 experiments. We used the method in this chapter to estimate the picking point and picking direction, and predicted the picking area of ​​138 leaves. Among them, there were 8 leaves in the unrecognized picking area, 3 of which were due to the picking area being blocked by small leaves on the edge of the leaves, resulting in recognition failure, and 5 were due to blurred shooting. The example segmentation result is shown in the figure Fig.10 , see the example diagram of tobacco leaf picking points Fig.11 , see the example diagram of tobacco leaf picking direction Fig.12 .

[0059] Experiments on picking point and picking direction estimation were carried out. The success rate in the unobstructed case was 94.52%, and the prediction success rate in the obstructed case was 91.40%. In addition, the polar angle and azimuth angle of the picking direction had small errors compared with the actual manual measurement.

[0060] The present invention provides a method for estimating the picking point and picking direction of tobacco leaves. The method adopts a new model construction and optimization strategy, performs instance segmentation of tobacco leaves based on the YOLACT++ algorithm, and combines point cloud data processing to estimate the picking point and picking direction of tobacco leaves in non-occluded situations. The system uses the Intel RealSense D435i camera to capture RGB and depth images of tobacco leaves, improves the image quality through a depth completion algorithm, and provides high-quality input for the model. In the backbone network of the model, a deformable convolutional layer DCN is used. DCN significantly improves the accuracy of the detection and segmentation model by using flexible sampling points instead of fixed grid sampling in traditional convolutional neural networks (CNN). Experimental results show that the present invention can better estimate the picking point and picking direction, and improve the automation level of tobacco leaf picking.

[0061] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the prior art. The computer software programs or protocols involved in the functional modules are themselves technologies that are well known to those skilled in the art and are not improvements of this system. The improvements of this system are the interaction or connection relationships between the modules, that is, improvements to the overall structure of the system to solve the corresponding technical problems to be solved by this system.

[0062] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for estimating tobacco leaf picking points and picking directions, characterized in that: The following steps are involved: Use the depth camera to obtain RGB and depth images, and preprocess the depth images using the depth completion algorithm; Obtain tobacco leaf images, calibrate the images, and obtain training data sets; Use YOLACT++ optimized with instance segmentation algorithm and point cloud data to identify and estimate picking points and picking directions in unobstructed situations.

2. A method for estimating tobacco leaf picking points and picking directions according to claim 1, characterized in that: The method for obtaining and preprocessing the depth image is as follows: Use Python language and depth measurement principle to capture color images, depth images and infrared images of Intel RealSense D435 i camera The camera is calibrated and the depth image is preprocessed using a depth completion algorithm.

3. A method for estimating tobacco leaf picking points and picking directions according to claim 2, characterized in that: The method of the depth measurement principle is as follows: The Intel RealSense D435 i depth camera combines the classic binocular vision method and infrared light projection technology to achieve accurate depth measurement. According to the principle of binocular imaging, the depth information of a point can be obtained by only knowing the parallax of that point.

4. A method for estimating tobacco leaf picking points and picking directions according to claim 2, characterized in that: The method of the calibration technique is as follows: Use Kalibr Visual-Inertial Calibration Toolbox to calibrate multiple cameras and transform image and IMU data into a common reference coordinate system Calibrate the IMU to eliminate the influence of errors on parameters during measurement The IMU is jointly calibrated with the RGB camera, and the IMU is jointly calibrated with the stereo camera to fuse the image data and the IMU data and align them to a common reference coordinate system.

5. The method for estimating tobacco leaf picking points and picking directions according to claim 2, characterized in that: The depth completion algorithm is as follows: The algorithm optimizes the depth image through steps such as depth inversion, kernel expansion, small hole closing and large hole filling to complete the missing depth values, which is suitable for the subsequent determination of picking points and directions.

6. A method for estimating tobacco leaf picking points and picking directions according to claim 2, characterized in that: The tobacco leaf picking point and picking direction estimation method is as follows: Calibration and training of tobacco leaf dataset; The picking area is determined to match the tobacco leaves according to a fixed threshold, and the tobacco plants are detected using the trained instance segmentation network to obtain the instance mask of the picking area, and the pixel center point of the picking area is calculated as the picking point; Use coordinate transformation and statistical filtering to preprocess the point cloud data and convert the depth map into a point cloud map; The camera's attitude is obtained through the IMU, and the point cloud is rotated so that the height in the real world is used as the value of the z-axis; PCA is used to estimate the normal vector of the picking area, and the picking direction is determined by the normal vector of the point cloud cluster near the picking point.

7. A method for estimating tobacco leaf picking points and picking directions according to claim 1, characterized in that: The instance segmentation algorithm is as follows: The instance segmentation algorithm used is the YOLACT++ algorithm, which improves the segmentation accuracy and speed in complex environments through an enhanced network structure and deformable convolutional layer DCN.

8. The method for estimating tobacco leaf picking points and picking directions according to claim 6, characterized in that: The method for matching tobacco leaves in the picking area is as follows: When a mature tobacco leaf that needs to be picked is photographed, it is necessary to calculate the ratio of the intersection area of ​​the tobacco leaf prediction frame and the prediction frame of each picking area to the picking area. If the ratio is greater than a set fixed threshold, it is determined that the picking area belongs to the tobacco leaf.

9. A method for estimating tobacco leaf picking points and picking directions according to claim 8, characterized in that: The ratio expression is as follows: Where: Area lsq1picking Area is the intersection area of ​​the prediction box of tobacco leaves and picking area; picking Predict the area of ​​the box for the picking area.

10. The method for estimating tobacco leaf picking points and picking directions according to claim 6, characterized in that: The point cloud processing technology is as follows: In order to reconstruct the exact position of the pixel value in the real world, a series of coordinate transformations must be performed to convert the depth map into a point cloud map, converting the points from the pixel coordinate system to the physical coordinate system, and then to the camera coordinate system; Statistical filtering is performed on the scattered noise points encountered in the data. The neighboring points of the statistical filter are set to 50, the threshold is set to 0.5σ, and the downsampling technology is used to reduce the amount of calculation.

11. A method for estimating tobacco leaf picking points and picking directions according to claim 6, characterized in that: The method of obtaining the camera attitude and rotating the point cloud is as follows: Use IMU to obtain the attitude of the Intel RealSense D435i camera. The yaw angle, pitch angle, and roll angle of the camera can be obtained by integration. The rotation matrix and the coordinates of the midpoint of the point cloud are used for matrix dot multiplication.

12. The method for estimating tobacco leaf picking points and picking directions according to claim 6, characterized in that: The expression of the rotation is as follows: Rotate counterclockwise around the x-axis by an angle θ x The formula is as follows: Rotate counterclockwise around the y-axis by an angle θ y The formula is as follows: Rotate counterclockwise around the z axis by an angle θ z The formula is as follows:

13. A method for estimating tobacco leaf picking points and picking directions according to claim 6, characterized in that: The PCA principal component analysis method is as follows: After the picking point of a specific picking area is determined, a spherical area with a radius of r centered at the picking point is obtained to form a point cloud set p = [p x ,p y ,p z ] T ; Find the mean of the coordinate system direction, decentralize the vectors in each direction to form a matrix p, find the covariance matrix of the matrix p and expand it to get the expression, apply the singular value decomposition algorithm to decompose the covariance matrix to get the expression; Project the vector into three-dimensional space and express it as the radius, polar angle and azimuth in the spherical coordinate system using axis-angle notation. Use the vector in the three-dimensional rectangular coordinate system to obtain the polar angle θ and azimuth Convert the normal vector obtained by PCA into polar angle θ and azimuth As a picking direction.

14. A method for estimating tobacco leaf picking points and picking directions according to claim 13, characterized in that: The expression is as follows: Find the covariance matrix of matrix p: Coν(p,p)=E[(pE(p)) 2 ]=E(p 2 ) Expand the covariance matrix, where the matrix p is decentralized, E(p) ＝0 : Apply the SVD (singular value decomposition) algorithm to decompose the covariance matrix: Three matrices can be obtained: U, ∑ and V. In the diagonal matrix ∑, the elements λ1, λ2, and λ3 on the diagonal are the eigenvalues ​​of the covariance matrix, and these eigenvalues ​​are arranged in order from large to small; in the V matrix, the corresponding three column vectors v1, ν2, and ν3 correspond to the eigenvectors of these eigenvalues ​​respectively; among them, the eigenvector v3 corresponding to the smallest eigenvalue λ3 is the normal vector at that point.

15. The method for estimating tobacco leaf picking points and picking directions according to claim 13, characterized in that: The PCA obtains the eigenvector as follows: The local plane H is first projected onto the eigenvector v1 to maximize the variance; then it is projected onto the eigenvector v2 to make the variance the second largest; and then when it is projected onto the third eigenvector v3, the eigenvector is the normal vector of the plane.

16. A method for estimating tobacco leaf picking points and picking directions according to claim 13, characterized in that: The axis-angle representation is as follows: In the spherical coordinate system, θ is the polar angle, which is the angle between the vector and the positive half axis of the z-axis, satisfying: 0≤θ≤π is the azimuth, which is the angle between the projection of the vector on the xoy plane and the positive x-axis, satisfying: The polar angle θ and azimuth angle can be obtained using the vector of the three-dimensional rectangular coordinate system The formula is as follows:

Images