Mechanical arm tobacco leaf picking path planning method based on improved RRT algorithm

Abstract

The invention relates to the technical field of agricultural machinery, and discloses a mechanical arm tobacco leaf picking path planning method based on an improved RRT algorithm, and the method comprises the following steps: S1, obtaining a tobacco plant image in a natural environment, and carrying out the marking processing of the obtained image; according to the mechanical arm tobacco leaf picking path planning method based on the improved RRT algorithm, in an operation environment obtained through point cloud preprocessing and octree modeling, the RRT algorithm is improved, and the improved RRT algorithm is combined with an RRT * algorithm and a Bi-RRT algorithm to form a Guing Bidirectional RRT * algorithm; in consideration of the fact that a path finally obtained by the GBi-RRT * algorithm is a broken line, a greedy algorithm is selected for optimization so as to reduce the length of the path; according to the method, a B spline curve is selected as a path smoothing method, the picking efficiency of the mechanical arm in the complex farmland environment is improved, meanwhile, the calculation cost of path planning is reduced, and compared with an existing RRT algorithm, the collision-free optimal path can be generated more rapidly and accurately, and the method has remarkable practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural machinery, and in particular to a method for planning a tobacco leaf picking path of a robotic arm based on an improved RRT algorithm. Background Art

[0002] In modern agriculture, robotic arms have been widely used in automatic harvesting. However, obstacles in complex farmland environments, such as tobacco leaves, often affect the harvesting efficiency of robotic arms. Current path planning methods, including grid maps, topological maps, and octree maps, each have their own advantages and disadvantages. For example, raster maps are easy to implement but require large amounts of computation; octree maps can efficiently organize three-dimensional data by recursively partitioning space, and are suitable for path planning in complex environments.

[0003] The rapidly exploring random tree (RRT) algorithm has become a commonly used path planning algorithm due to its ability to quickly generate collision-free paths. However, the high randomness of the RRT algorithm may cause a lot of time to be spent in non-target areas, affecting planning efficiency. Therefore, this paper proposes the high randomness of the RRT algorithm to solve the above problem. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the shortcomings of the prior art, the present invention provides a robotic arm tobacco picking path planning method based on an improved RRT algorithm, which has the advantages of improving the efficiency and accuracy of the prior art in complex environments, and solves the problem that the high randomness of the existing RRT algorithm may cause a lot of time to be consumed in non-target areas, affecting the planning efficiency.

[0005] (II) Technical solution To achieve the above object, the present invention provides the following technical solution: a method for planning a tobacco leaf picking path for a robot arm based on an improved RRT algorithm, comprising the following steps: S1, obtaining an image of a tobacco plant in a natural environment, and annotating the obtained image, then obtaining an instance segmentation result of the tobacco plant and obtaining a picking point based on the instance segmentation result; S2, converting the acquired depth map into a three-dimensional point cloud, preprocessing the obtained point cloud, and then performing octree modeling; S3. Improve the RRT algorithm, name the improved algorithm Guiding RRT and combine it with the RRT* and Bi-RRT algorithms to form the Guiding Bidirectional RRT* algorithm, and then conduct comparative experiments; S4, using a greedy algorithm to optimize the path obtained in step 3 to reduce the distance, and smoothing the obtained path; S5. Conduct path planning experiments on Guiding RRT, Guiding RRT* and GBi-RRT* mentioned in step 3, conduct experiments on path optimization and smoothing after the path is generated by the GBi-RRT* algorithm, and finally compare the method of the present invention with the RRT algorithm.

[0006] Preferably, the specific steps of S1 are as follows: S1.1. Use Intel RealSense D435i depth camera to obtain tobacco plant depth map and perform depth completion on the captured depth image; S1.2. Use the polygonal box in the "labelme" annotation tool to select along the edge of the target. There are four labels: tobacco leaves, picking area, tobacco leaf main veins, and tobacco plant trunks. S1.3, training the model to obtain tobacco plant instance segmentation results; S1.4. For unobstructed tobacco leaves, the instance mask of the picking area can be directly obtained, and the pixel center point of the picking area is calculated as the picking point, and the picking point is obtained through the instance segmentation result.

[0007] Preferably, the specific steps of S2 are as follows: S2.1. Use the depth and RGB images acquired by the Intel RealSense D435i camera and calibrate them, convert the depth data into 3D coordinate points, and use the color data in the RGB image to color these points to obtain a colored 3D point cloud with the same resolution as the depth map. Objects beyond the camera's measurement range will be defined as 0 depth, and these invalid depth values ​​will be excluded. S2.2. Filter the acquired point cloud using statistical filtering. By setting the parameters of the statistical filter, especially selecting 50 neighboring points and setting the threshold to 0.5, effective noise point removal is achieved, and the valid point cloud required for the experiment is retained. In addition, in the face of the high computational cost problem caused by too many point clouds, voxel downsampling technology is used to reduce the amount of calculation; S2.3. Use the preprocessed point cloud to perform octree modeling on the point cloud of the tobacco plant.

[0008] Preferably, the specific steps of S3 are as follows: S3.1, introduce heuristic information to enhance the guidance of the target point of the RRT algorithm. When the algorithm expands the random tree and adds new nodes, it Let the newly added leaf nodes lean towards the target point; S3.2. For different probabilities Experiments were conducted, with 3 planning paths set up, and each path was planned 10 times on average, for each probability Conduct 30 path planning experiments, set the maximum number of sampling times to 3000, and count the average number of sampling times and path distance. According to the experimental results, select the probability is 0.1; S3.3, increase the deviation of the sampling point towards the target point. When the sampling point is obtained, deflect it towards the target point; S3.4, combining the Guiding RRT algorithm with the RRT* and Bi-RRT algorithms into the Guiding Bidirectional RRT* algorithm; S3.5. Conduct path planning experiments on Guiding RRT, Guiding RRT*, and Guiding Bidirectional RRT* respectively, and obtain the path planning results of the three algorithms.

[0009] Preferably, the specific steps of S4 are as follows: S4.1, a greedy algorithm-based strategy is used to optimize the existing route, starting from the node Start from here and consider the connections of subsequent nodes one by one; S4.2, path smoothing experiments were conducted on two common smoothing methods: Bezier curve and B-spline curve; S4.3. Based on the experimental results obtained in step S4.2, the B-spline curve is selected to optimize the algorithm path.

[0010] Preferably, the specific steps of S5 are as follows: S5.1. Set up three picking paths, conduct 10 experiments on each path on average, conduct 30 path planning experiments on Guiding RRT, Guiding RRT* and GBi-RRT* respectively, set the maximum number of sampling times to 3000 times, and calculate the average number of sampling times and path distance; S5.2, conduct experiments on path optimization and smoothing after the path is generated by GBi-RRT* algorithm, the experimental content is the same as S1.1; S5.3. Compare the method of the present invention with the RRT algorithm.

[0011] (III) Beneficial effects Compared with the prior art, the present invention provides a robot arm tobacco leaf picking path planning method based on an improved RRT algorithm, which has the following beneficial effects: The robot arm tobacco leaf picking path planning method based on the improved RRT algorithm improves the RRT algorithm under the operating environment obtained by point cloud preprocessing and octree modeling, and combines it with the RRT* and Bi-RRT algorithms to form a Guiding Bidirectional RRT* algorithm; considering that the path finally obtained by the GBi-RRT* algorithm is a broken line, a greedy algorithm is selected for optimization to reduce the distance length; and a B-spline curve is selected as a path smoothing method. This method not only improves the picking efficiency of the robot arm in a complex farmland environment, but also reduces the calculation cost of path planning. Compared with the existing RRT algorithm, the present invention can generate a collision-free optimal path more quickly and accurately, and has significant practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is the segmentation and picking point marking result diagram of the present invention; Figure 2 This is the result graph of converting point cloud to octree in the present invention; Figure 3 This is a graph showing the experimental results of different probability p path planning of the improved RRT algorithm of the present invention; Figure 4 This is the Guiding RRT path planning result diagram of the present invention; Figure 5 This is the Guiding RRT* path planning result diagram of the present invention; Figure 6 This is the Guiding Bidirectional RRT* path planning result diagram of the present invention; Figure 7 This is a Bezier curve path smoothing effect diagram of the present invention; Figure 8 This is a B-spline curve path smoothing effect diagram of the present invention; Fig. 9 It is the path optimization graph of the B-spline curve path algorithm of the present invention; Fig.10 Figure 1 is a graph showing the path planning experiments of Guiding RRT, Guiding RRT* and GBi-RRT* algorithms of the present invention; Fig.11 This is a comparison diagram of the path optimization and smoothing experiments after the GBi-RRT* algorithm of the present invention generates the path; Fig.12 It is a comparison chart between the method of the present invention and the RRT algorithm. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0014] See also Figure 1-12 , a robot arm tobacco leaf picking path planning method based on improved RRT algorithm, including the following steps: S1, obtaining an image of a tobacco plant in a natural environment, and annotating the obtained image, then obtaining an instance segmentation result of the tobacco plant and obtaining a picking point based on the instance segmentation result; S2, converting the acquired depth map into a three-dimensional point cloud, preprocessing the obtained point cloud, and then performing octree modeling; S3. Improve the RRT algorithm, name the improved algorithm Guiding RRT and combine it with the RRT* and Bi-RRT algorithms to form the Guiding Bidirectional RRT* algorithm, and then conduct comparative experiments; S4, using a greedy algorithm to optimize the path obtained in step 3 to reduce the distance, and smoothing the obtained path; S5. Conduct path planning experiments on Guiding RRT, Guiding RRT* and GBi-RRT* mentioned in step 3, conduct experiments on path optimization and smoothing after the path is generated by the GBi-RRT* algorithm, and finally compare the method of the present invention with the RRT algorithm.

[0015] Specifically, the specific steps of S1 are as follows: S1.1. Use Intel RealSense D435i depth camera to obtain tobacco plant depth map and perform depth completion on the captured depth image; S1.2. Use the polygonal box in the "labelme" annotation tool to select along the edge of the target. There are four labels: tobacco leaves, picking area, tobacco leaf main veins, and tobacco plant trunks. S1.3, training the model to obtain tobacco plant instance segmentation results; S1.4. For unobstructed tobacco leaves, the instance mask of the picking area can be directly obtained, and the pixel center point of the picking area is calculated as the picking point, and the picking point is obtained through the instance segmentation result.

[0016] Specifically, the specific steps of S2 are as follows: S2.1. Use the depth and RGB images acquired by the Intel RealSense D435i camera and calibrate them, convert the depth data into 3D coordinate points, and use the color data in the RGB image to color these points to obtain a colored 3D point cloud with the same resolution as the depth map. Objects beyond the camera's measurement range will be defined as 0 depth, and these invalid depth values ​​will be excluded. S2.2. Filter the acquired point cloud using statistical filtering. By setting the parameters of the statistical filter, especially selecting 50 neighboring points and setting the threshold to 0.5, effective noise point removal is achieved, and the valid point cloud required for the experiment is retained. In addition, in the face of the high computational cost problem caused by too many point clouds, voxel downsampling technology is used to reduce the amount of calculation; S2.3. Use the preprocessed point cloud to perform octree modeling on the point cloud of the tobacco plant.

[0017] Specifically, S3.1 introduces heuristic information to enhance the guidance of the target point of the RRT algorithm. When the algorithm expands the random tree and adds new nodes, it Let the newly added leaf nodes be biased towards the target point. The improved node generation formula is as follows:

[0018] in: is the target point; is the probability of leaning toward the target point, when Less than When As Used to determine the new generation point, otherwise select As Used to determine the new generation point, node It is randomly generated; S3.2. For different probabilities The experiment was carried out and the experimental results are shown in Figure 3 , set 3 planning paths, and perform 10 path plannings on each path on average, for each probability Conduct 30 path planning experiments, set the maximum number of sampling times to 3000, and count the average number of sampling times and path distance. According to the experimental results, select the probability is 0.1; S3.3, increase the deviation of the sampling point to the target point. When the sampling point is obtained, it is deflected toward the target point. This is conducive to rapid convergence to the target. Add the following optimizations:

[0019] in: is the target point; is the starting point; is the coefficient to prevent over-correction, and is taken as 0.5 in this algorithm; is the attenuation factor, which decays as the number of sampling increases; is the random sampling number; The compensation failure round is set to prevent the inability to find a valid path due to obstacles after adding the target point deviation; S3.4, combining the Guiding RRT algorithm with the RRT* and Bi-RRT algorithms into the Guiding Bidirectional RRT* algorithm; S3.5. Conduct path planning experiments on Guiding RRT, Guiding RRT*, and Guiding Bidirectional RRT* respectively, and obtain the path planning results of the three algorithms.

[0020] Specifically, S4.1 adopts a greedy algorithm-based strategy to optimize the existing route, starting from node Start from, and consider the connection of subsequent nodes one by one; during the connection process, if If there is no obstacle between the node and the subsequent node, the connection will continue. arrive When an obstacle appears on the path of arrive The path is used as a new connection, replacing the original path until All connections, then, with As a new starting point, continue to perform similar operations until the target node is finally reached. .

[0021] S4.2. Conduct path smoothing experiments on two common smoothing methods: Bezier curve and B-spline curve: Bezier curve: The mathematical basis of the Bezier curve is the Bernstein polynomial. +1 position vector, we can define The order Bezier curve is:

[0022] in: is the sampling point in the path; is the position ratio of the point in the curve, When it is 0, it is the starting point of the curve. When it is 1, it is the end point of the curve; is the Bernstein polynomial.

[0023] The expression is as follows:

[0024] Use Bezier curve to optimize the path trajectory, and its smoothing effect can be seen Figure 7 , the light-colored curve is the smooth path, the light-colored points are the control points, and the dark-colored lines are the original path.

[0025] B-spline curve: B-spline curve is based on Bezier curve and improves the above problems. B-spline curve is a commonly used method in path smoothing. It optimizes the path by locally reconstructing the original path. It has locality. Similar to Bezier curve, B-spline curve is determined by B-spline basis function. +1 control point, with a knot vector , The sub-B-spline curve is expressed as:

[0026] in: are the characteristic polygon vertices that constitute the B-spline; is the B-spline basis function.

[0027] Using the Cox-deBoor recursive formula:

[0028] in: is a node vector, and ; is the degree of the B-spline curve.

[0029] Basis Functions In the interval is non-zero on the interval, because there is always a zero-order basis function that is not zero on this interval. , this interval is called the support space, and the corresponding segment on the spline curve is the support curve. In order to make each control node matched by the support interval, the number of nodes should be ,This paper uses the uniform method to generate nodes. It is necessary to ensure that the curve passes through the starting point and the end point. Therefore, the first and last node repetition of the node vector is set to ( =4), the node generation formula is as follows:

[0030] Generally, the higher the order of a curve, the higher the order of its derivative, which results in the curve having zero values ​​at multiple points, that is, there are multiple extreme points, forming multiple peaks and troughs. Therefore, a spline curve with a lower order can more closely approximate the control point. On the other hand, the cubic B-spline curve can ensure the continuity of the second-order derivative, so this study mainly uses the cubic B-spline curve to smooth the path. Using the Cox-deBoor recursion formula, the equation of the cubic B-spline curve can be derived as follows:

[0031] According to the above formula, we can get the points on the B-spline curve and thus get the final smoothed path. The smoothing effect of the B-spline curve can be seen in Figure 8 , the light-colored curve is the smooth path, the light-colored points are the control points, and the dark-colored lines are the original path; S4.3. Based on the experimental results obtained in step S4.2, the B-spline curve is selected to optimize the algorithm path. The results are shown in Fig. 9 .

[0032] Specifically, the specific steps of S5 are as follows: S5.1. Set up three picking paths, conduct 10 experiments on each path on average, conduct 30 path planning experiments on Guiding RRT, Guiding RRT* and GBi-RRT* respectively, set the maximum sampling number to 3000 times, and count the average sampling number and path distance. The experimental results are shown in Fig.10 .

[0033] According to the results, when the average sampling times of Guiding RRT* are similar to those of Guiding RRT, the average path length is reduced by 24.05%, which is a significant improvement in path length. Compared with Guiding RRT, the average sampling times of GBi-RRT* are reduced by 52.95%, and the average path length is reduced by 28.02%. At the same time, all 30 path plannings are successful. GBi-RRT* can greatly improve the speed of path planning and increase the quality of generated paths.

[0034] S5.2. Experiments were conducted on path optimization and smoothing after the GBi-RRT* algorithm generated the path. The experimental contents were the same as S5.1. The experimental results are shown in Fig.11 .

[0035] It can be found that the addition of greedy thinking can reduce the average path length by 5.63%, and the optimization effect on the path is more obvious. After the greedy thinking, the path is smoothed by B-spline curve, which can reduce the average path length by 12.3%, which can greatly reduce the moving distance.

[0036] S5.3. The method of the present invention is compared with the RRT algorithm. The experimental results are shown in Fig.12 .

[0037] It can be seen from the results that based on the RRT algorithm, the average sampling times of the proposed algorithm are reduced by 83.18%, the planning success rate is increased by 16.67%, and the average path length is reduced by 55.24%, which is a huge improvement compared with the RRT algorithm.

[0038] In summary, the robot arm tobacco leaf picking path planning method based on the improved RRT algorithm, the robot arm tobacco leaf picking path planning method based on the improved RRT algorithm, under the operating environment obtained by point cloud preprocessing and octree modeling, the RRT algorithm is improved, and combined with the RRT* and Bi-RRT algorithms to form a Guiding Bidirectional RRT* algorithm; considering that the path finally obtained by the GBi-RRT* algorithm is a broken line, the greedy algorithm is selected for optimization to reduce the distance length; and the B-spline curve is selected as the path smoothing method. This method not only improves the picking efficiency of the robot arm in a complex farmland environment, but also reduces the calculation cost of path planning. Compared with the existing RRT algorithm, the present invention can generate a collision-free optimal path more quickly and accurately, has significant practical value, and solves the problem that the high randomness of the existing RRT algorithm may cause a lot of time to be spent in non-target areas, affecting the planning efficiency.

[0039] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0040] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for planning the path of tobacco leaf picking by a robotic arm based on an improved RRT algorithm, characterized in that: The following steps are involved: S1, obtaining an image of a tobacco plant in a natural environment, and annotating the obtained image, then obtaining an instance segmentation result of the tobacco plant and obtaining a picking point based on the instance segmentation result; S2, converting the acquired depth map into a three-dimensional point cloud, preprocessing the obtained point cloud, and then performing octree modeling; S3. Improve the RRT algorithm, name the improved algorithm Guiding RRT and combine it with the RRT* and Bi-RRT algorithms to form the Guiding Bidirectional RRT* algorithm, and then conduct comparative experiments; S4, using a greedy algorithm to optimize the path obtained in step 3 to reduce the distance, and smoothing the obtained path; S5. Conduct path planning experiments on Guiding RRT, Guiding RRT* and GBi-RRT* mentioned in step 3, conduct experiments on path optimization and smoothing after the path is generated by the GBi-RRT* algorithm, and finally compare the method of the present invention with the RRT algorithm.

2. According to claim 1, a method for planning a path for tobacco leaf picking by a robot arm based on an improved RRT algorithm is characterized in that: The specific steps of S1 are as follows: S1.

1. Use Intel RealSense D435i depth camera to obtain tobacco plant depth map and perform depth completion on the captured depth image; S1.

2. Use the polygonal box in the "labelme" annotation tool to select along the edge of the target. There are four labels: tobacco leaves, picking area, tobacco leaf main veins, and tobacco plant trunks. S1.3, training the model to obtain tobacco plant instance segmentation results; S1.

4. For unobstructed tobacco leaves, the instance mask of the picking area can be directly obtained, and the pixel center point of the picking area is calculated as the picking point, and the picking point is obtained through the instance segmentation result.

3. According to claim 1, a robot arm tobacco leaf picking path planning method based on an improved RRT algorithm is characterized in that: The specific steps of S2 are as follows: S2.

1. Use the depth and RGB images acquired by the Intel RealSense D435i camera and calibrate them, convert the depth data into 3D coordinate points, and use the color data in the RGB image to color these points to obtain a colored 3D point cloud with the same resolution as the depth map. Objects beyond the camera's measurement range will be defined as 0 depth, and these invalid depth values ​​will be excluded. S2.

2. Filter the acquired point cloud using statistical filtering. By setting the parameters of the statistical filter, especially selecting 50 neighboring points and setting the threshold to 0.5, effective noise point removal is achieved, and the valid point cloud required for the experiment is retained. In addition, in the face of the high computational cost problem caused by too many point clouds, voxel downsampling technology is used to reduce the amount of calculation; S2.

3. Use the preprocessed point cloud to perform octree modeling on the point cloud of the tobacco plant.

4. According to claim 1, a method for planning a path for tobacco leaf picking by a robot arm based on an improved RRT algorithm is characterized in that: The specific steps of S3 are as follows: S3.1, introduce heuristic information to enhance the guidance of the target point of the RRT algorithm. When the algorithm expands the random tree and adds new nodes, it Let the newly added leaf nodes lean towards the target point; S3.

2. For different probabilities Experiments were conducted, with 3 planning paths set up, and each path was planned 10 times on average, for each probability Conduct 30 path planning experiments, set the maximum number of sampling times to 3000, and count the average number of sampling times and path distance. According to the experimental results, select the probability is 0.1; S3.3, increase the deviation of the sampling point towards the target point. When the sampling point is obtained, deflect it towards the target point; S3.4, combining the Guiding RRT algorithm with the RRT* and Bi-RRT algorithms into the Guiding Bidirectional RRT* algorithm; S3.

5. Conduct path planning experiments on Guiding RRT, Guiding RRT*, and Guiding Bidirectional RRT* respectively, and obtain the path planning results of the three algorithms.

5. According to claim 1, a method for planning a path for tobacco leaf picking by a robot arm based on an improved RRT algorithm is characterized in that: The specific steps of S4 are as follows: S4.1, a greedy algorithm-based strategy is used to optimize the existing route, starting from the node Start from here and consider the connections of subsequent nodes one by one; S4.2, path smoothing experiments were conducted on two common smoothing methods: Bezier curve and B-spline curve; S4.

3. Based on the experimental results obtained in step S4.2, the B-spline curve is selected to optimize the algorithm path.

6. According to claim 1, a robot arm tobacco leaf picking path planning method based on an improved RRT algorithm is characterized in that: The specific steps of S5 are as follows: S5.

1. Set up three picking paths, conduct 10 experiments on each path on average, conduct 30 path planning experiments on Guiding RRT, GuidingRRT* and GBi-RRT* respectively, set the maximum number of sampling times to 3000 times, and calculate the average number of sampling times and path distance; S5.2, conduct experiments on path optimization and smoothing after the path is generated by GBi-RRT* algorithm, the experimental content is the same as S1.1; S5.

3. Compare the method of the present invention with the RRT algorithm.

Images