Path planning method, device and medium for flower pollination

Abstract

The embodiments of the present specification disclose a path planning method, device and medium for flower pollination, which relate to the field of flower pollination technology. The method includes: collecting point cloud data of plants to be pollinated for semantic segmentation to generate style position data; determining style posture data based on the style position data through a point pair feature method, and determining the stamen orientation information of each style to be pollinated; determining the pollination path planning end point and a pre-pollination point based on the stamen orientation information, calculating the distance between the pre-pollination point and the pollination path planning end point, and when the distance is greater than a preset distance setting value, constructing a reference coordinate system through the pre-pollination point and the pollination path planning end point; determining a leaf obstacle set in the reference coordinate system, and establishing a sampling space based on the leaf obstacle set in a specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through a BI-RRT algorithm of an improved artificial potential field method, thereby realizing pollination of the plants to be pollinated.

Description

Technical Field

[0001] This specification relates to the technical field of flower pollination, and in particular to a path planning method, device, and medium for flower pollination. Background Art

[0002] In the crop planting process, pollination is necessary to increase crop yields. Since flowers are mostly obscured by leaves, artificial pollination has become a preferred solution. However, with the development of society, manual pollination no longer meets social needs. Faced with a large number of plants to be pollinated, artificial pollination requires a lot of manpower and pollination efficiency cannot be guaranteed. With the continuous development of intelligent planting, the integration of intelligent robots with agricultural production will further promote the development of precision agriculture. It is an essential requirement for future agricultural development. It will not only greatly reduce labor output, but also greatly improve the level of intelligence and precision, effectively improving agricultural production efficiency and product quality.

[0003] When using robots to automatically pollinate flowers, a pollination path needs to be planned. Generally, a rapidly exploring random tree (RRT) algorithm is used for path planning. However, the postures and leaf distributions of different flowers vary, and the RRT algorithm is too random, resulting in low path planning efficiency. As a result, the planned path cannot meet actual pollination needs. Summary of the Invention

[0004] One or more embodiments of this specification provide a path planning method, device, and medium for flower pollination, which are used to solve the following technical problems: the postures and leaf conditions of different flowers are distributed differently, and the RRT algorithm is too random, resulting in low path planning efficiency and the planned path cannot meet actual pollination needs.

[0005] One or more embodiments of this specification adopt the following technical solutions:

[0006] One or more embodiments of the present specification provide a path planning method for flower pollination, the method comprising: collecting point cloud data of plants to be pollinated, performing semantic segmentation on the point cloud data of plants to be pollinated, and generating style position data of multiple styles to be pollinated of the plants to be pollinated; determining style posture data of each style to be pollinated based on the style position data of each style to be pollinated by a point pair feature method, and determining the stamen orientation information of each style to be pollinated based on the style posture data; determining the pollination path planning end point and the end point based on the stamen orientation information of each style to be pollinated A pre-pollination point is used to calculate the distance between the pre-pollination point and the end point of the pollination path planning. When the distance is greater than a preset distance setting value, a reference coordinate system is constructed through the pre-pollination point and the end point of the pollination path planning; in the reference coordinate system, a set of leaf obstacles in a specified area corresponding to the pre-pollination point and the end point of the pollination path planning is determined, and a sampling space is established based on the set of leaf obstacles in the specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

[0007] Furthermore, based on the stamen orientation information of each style to be pollinated, the pollination path planning end point and the pre-pollination point are determined, specifically including: pre-constructing a virtual cylindrical structure corresponding to each style to be pollinated, wherein the virtual cylindrical structure includes the style radius; according to the stamen orientation information of each style to be pollinated, determining a designated bottom surface corresponding to the stamen orientation information in the virtual cylindrical structure, and taking the center of the designated bottom surface as the pollination path planning end point; taking the pollination path planning end point as the starting point and the pre-acquired initial pollination starting point as the end point, each Generate multiple reference pre-pollination points in sequence at specified distances in a specified direction, wherein the specified direction is related to the pistil orientation information of each style to be pollinated; determine leaf obstacle information between the multiple reference pre-pollination points and a predetermined pollination starting point in sequence, wherein the leaf obstacle information includes the number of leaf obstacles; based on the leaf obstacle information, determine a pre-pollination point that meets the requirements among the multiple reference pre-pollination points, wherein the leaf obstacle information corresponding to the pre-pollination point that meets the requirements is that the number of leaf obstacles is less than a preset number threshold.

[0008] Furthermore, a reference coordinate system is constructed through the pre-pollination point and the pollination path planning end point, specifically including: connecting the pre-pollination point and the pollination path planning end point as the X-axis, and setting the positive direction of the X-axis as the direction of the pollination path planning end point specified by the pre-pollination point; based on the X-axis, determining a first plane containing the X-axis and perpendicular to the ground; determining the midpoint of the line connecting the pre-pollination point and the pollination path planning end point in the first plane, so as to construct a Y-axis based on the midpoint, and the positive direction of the Y-axis is the growth direction of the plant to be pollinated, wherein the Y-axis belongs to the first plane and is perpendicular to the X-axis; based on the X-axis and the Y-axis, determining the Z-axis, and setting the positive direction of the Z-axis to the designated direction.

[0009] Furthermore, in the reference coordinate system, a set of leaf obstacles within the designated area corresponding to the pre-pollination point and the end point of the pollination path planning is determined, specifically including: obtaining a pre-constructed virtual cylindrical structure corresponding to each style to be pollinated, determining the style radius in the virtual cylindrical structure, and generating a search radius based on the style radius; using the line between the pre-pollination point and the end point of the pollination path planning as the axis, generating a cylindrical leaf obstacle search area according to the search radius; within the cylindrical leaf obstacle search area, performing leaf obstacle search in a preset order to generate an ordered leaf obstacle set.

[0010] Furthermore, a sampling space is established based on the set of leaf obstacles in the designated area, specifically including: in the ordered set of leaf obstacles, coordinate analysis is performed on each leaf obstacle in turn based on the reference coordinate system to obtain multiple leaf point position coordinates corresponding to each leaf obstacle; based on the multiple leaf point position coordinates, multiple local optional points are determined from the multiple leaf points of each leaf obstacle, wherein the local optional points include any one or more of the leaf point corresponding to the maximum Z value, the leaf point corresponding to the minimum Z value, the leaf point corresponding to the maximum Y value, and the leaf point corresponding to the minimum Y value; the distance between each of the local optional points and the pre-pollination point or the pollination path planning end point is determined, and an ordered subset of local optional points corresponding to each leaf obstacle is generated in order of distance from small to large; the sampling space is established through the ordered subset of local optional points corresponding to each leaf obstacle.

[0011] Furthermore, the sampling space is established through the ordered subset of local optional points corresponding to each leaf obstacle, specifically including: determining at least one local optional point, at least one local line segment between the pre-pollination point and the pollination path planning end point based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point and the pollination path planning end point; generating a local cylinder corresponding to each of the local line segments to obtain multiple local cylinders; determining the overlapping areas between the multiple local cylinders, and superimposing the other areas of the multiple local cylinders except the overlapping areas to obtain the sampling space.

[0012] Furthermore, based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point and the planned end point of the pollination path, at least one local optional point and at least one local line segment between the pre-pollination point and the planned end point of the pollination path are determined, specifically including: determining the first leaf obstacle in the ordered set of leaf obstacles to determine the first ordered subset of local optional points corresponding to the first leaf obstacle; determining the first local optional point in the ordered subset of the first local optional point, performing a connection judgment on the first local optional point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the first local optional point and the pre-pollination point; if the leaf obstacle exists between the lines, obtaining a second local optional point in the ordered subset of the first local optional points that is located after the first local optional point; performing a connection judgment on the second local optional point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the second local optional point and the pre-pollination point; if there is no leaf obstacle between the lines, performing a connection judgment on the second local optional point and the pollination path planning end point to determine whether there is a leaf obstacle between the second local optional point and the pollination path planning end point; when there is no leaf obstacle between the lines, generating a first local line segment between the pre-pollination point and the second local optional point, and generating a second local line segment between the second local optional point and the pollination path planning end point.

[0013] Furthermore, after determining whether there is a leaf obstacle between the second local optional point and the end point of the pollination path planning, the method also includes: when the leaf obstacle exists between the connecting lines, determining the second leaf obstacle located after the first leaf obstacle in the ordered leaf obstacle set to determine the second local optional point ordered subset corresponding to the second leaf obstacle; performing a connection judgment on the designated local optional point and the second local optional point that are first in the ordered subset of the second local optional points to determine whether there is a leaf obstacle between the connecting line between the designated local optional point and the second local optional point; if there is no leaf obstacle between the connecting line between the designated local optional point and the second local optional point, determining whether there is a leaf obstacle between the designated local optional point and the end point of the pollination path planning; when there is no leaf obstacle between the connecting lines, generating a first local line segment between the pre-pollination point and the second local optional point, generating a second designated local line segment between the second local optional point and the designated local optional point, and generating a third local line segment between the designated local optional point and the end point of the pollination path planning.

[0014] One or more embodiments of this specification provide a path planning device for flower pollination, including:

[0015] at least one processor; and,

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: collect point cloud data of the plant to be pollinated to perform semantic segmentation on the point cloud data of the plant to be pollinated, and generate style position data of multiple styles to be pollinated of the plant to be pollinated; determine the style posture data of each style to be pollinated based on the style position data of each style to be pollinated by a point pair feature method, and determine the stamen orientation information of each style to be pollinated based on the style posture data; determine the stamen orientation information of each style to be pollinated based on the stamen orientation information of each style to be pollinated The pollination path planning end point and the pre-pollination point are calculated, and the distance between the pre-pollination point and the pollination path planning end point is calculated. When the distance is greater than a preset distance setting value, a reference coordinate system is constructed through the pre-pollination point and the pollination path planning end point; in the reference coordinate system, a set of leaf obstacles in a specified area corresponding to the pre-pollination point and the pollination path planning end point is determined, and a sampling space is established according to the set of leaf obstacles in the specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

[0018] One or more embodiments of this specification provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to:

[0019] Collect the point cloud data of the plants to be pollinated to perform semantic segmentation on the point cloud data of the plants to be pollinated, and generate the style position data of the multiple styles to be pollinated of the plants to be pollinated; determine the style posture data of each style to be pollinated according to the style position data of each style to be pollinated by the point pair feature method, and determine the stamen orientation information of each style to be pollinated based on the style posture data; determine the end point and pre-pollination point of the pollination path planning based on the stamen orientation information of each style to be pollinated, and calculate the distance between the pre-pollination point and the pollination path. When the distance between the pre-pollination point and the pollination path planning end point is greater than a preset distance setting value, a reference coordinate system is constructed through the pre-pollination point and the pollination path planning end point; in the reference coordinate system, a set of leaf obstacles in a specified area corresponding to the pre-pollination point and the pollination path planning end point is determined, and a sampling space is established according to the set of leaf obstacles in the specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

[0020] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects: through the above technical solution, the position of the style is obtained from the point cloud data of the plant to be pollinated, the current state of the style is analyzed to obtain the stamen orientation information, and the pollination path planning end point and the pre-pollination point are determined according to the stamen orientation, that is, the starting point and end point in the pollination path are combined with the real-time state of the current plant to be pollinated, ensuring the accuracy and pertinence of the path planning start point and end point, and can adapt to the postures of multiple plants; when the distance between the pre-pollination point and the pollination path planning end point is greater than the preset distance setting value, the reference coordinates are constructed in combination with the pre-pollination point and the pollination path planning end point. In the reference coordinate system, leaf obstacle analysis is performed to obtain a sampling space, and the leaf obstacles are analyzed in combination with the leaf distribution of the plant, thereby ensuring the pertinence of the analysis results. In addition, the sampling space is obtained in combination with the leaf distribution of the plant and the direction of the stamens, thereby ensuring the preparation work before path planning and providing a sampling space without leaf obstacles. By improving the BI-RRT algorithm of the artificial potential field method, a pollination path corresponding to each style to be pollinated is generated in the established sampling space, which can solve the problem of excessive randomness of the RRT algorithm, improve the efficiency of path planning, and meet the actual pollination needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are only some of the embodiments described in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without inventive work. In the drawings:

[0022] Figure 1 A schematic flow chart of a path planning method for flower pollination provided in an embodiment of this specification;

[0023] Figure 2 A schematic diagram of a reference coordinate system provided in an embodiment of this specification;

[0024] Figure 3 A schematic diagram of generating a local line segment provided in an embodiment of this specification;

[0025] Figure 4 This is a structural diagram of a path planning device for flower pollination provided in an embodiment of this specification. DETAILED DESCRIPTION

[0026] To help those skilled in the art better understand the technical solutions in this specification, the following will provide a clear and complete description of the technical solutions in the embodiments of this specification, in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this specification without creative work should fall within the scope of protection of this specification.

[0027] In the crop planting process, pollination is necessary to increase crop yields. Since flowers are mostly obscured by leaves, artificial pollination has become a preferred solution. However, with the development of society, manual pollination no longer meets social needs. Faced with a large number of plants to be pollinated, artificial pollination requires a lot of manpower and pollination efficiency cannot be guaranteed. With the continuous development of intelligent planting, the integration of intelligent robots with agricultural production will further promote the development of precision agriculture. It is an essential requirement for future agricultural development. It will not only greatly reduce labor output, but also greatly improve the level of intelligence and precision, effectively improving agricultural production efficiency and product quality.

[0028] When using robots to automatically pollinate flowers, a pollination path needs to be planned. Generally, a rapidly exploring random tree (RRT) algorithm is used for path planning. However, the postures and leaf distributions of different flowers vary, and the RRT algorithm is too random, resulting in low path planning efficiency. As a result, the planned path cannot meet actual pollination needs.

[0029] The embodiment of this specification provides a path planning method for flower pollination. It should be noted that the execution subject in the embodiment of this specification can be a server or any device with data processing capabilities, and can be applied to a mobile pollination robot with binocular vision. Figure 1 A flow chart of a path planning method for flower pollination provided in an embodiment of this specification is shown as follows: Figure 1 As shown, it mainly includes the following steps:

[0030] Step S101 : collecting point cloud data of the plants to be pollinated, performing semantic segmentation on the point cloud data of the plants to be pollinated, and generating style position data of a plurality of styles to be pollinated of the plants to be pollinated.

[0031] In one embodiment of the present specification, a pollination assembly is used to automatically pollinate a plant to be pollinated. The pollination assembly includes a mobile robot, a boom, and a truss. The mobile robot includes a baffle, a boom cantilever, a binocular camera, and a robotic arm. The location of the plant to be pollinated is determined, and the mobile pollination robot is moved directly above the plant. The boom is controlled to be lowered so that the plant to be pollinated is covered by the pollination robot's baffle. The baffle is expanded to isolate the branches and leaves of adjacent plants from the target plant area, and the branches and leaves of the target plant are fully extended. The pollination robot completes a surrounding point cloud capture around the boom, obtains a complete point cloud of the plant to be pollinated, and obtains point cloud data of the plant to be pollinated.

[0032] The pre-trained PointRCNN 3D recognition model is used to identify and semantically segment the acquired point cloud data of the plants to be pollinated. This determines the positions of all the flowers and styles to be pollinated in the world coordinate system, i.e., the style position data. The pollination order for all flowers to be pollinated is determined based on the style position data of each style. This pollination order can be determined based on the pollination method combined with empirical data, such as a top-down pollination order. This allows for sequential pollination according to the determined pollination order. The pollination machine rotates to allow the robotic arm to reach the vicinity of the flowers to be pollinated.

[0033] Step S102, using a point pair feature method, according to the style position data of each style to be pollinated, the style posture data of each style to be pollinated is determined, so as to determine the stamen orientation information of each style to be pollinated based on the style posture data.

[0034] In one embodiment of the present specification, a point-pair feature method is used to determine the style position data of each style to be pollinated based on the style position data of each style to be pollinated. Based on the style position data, the stamen orientation information of each style to be pollinated is determined. The stamen orientation information here is used to indicate the current orientation of the stamen, such as upward orientation, downward orientation, etc.

[0035] Step S103, based on the pistil orientation information of each style to be pollinated, determine the pollination path planning end point and the pre-pollination point, calculate the distance between the pre-pollination point and the pollination path planning end point, and when the distance is greater than the preset distance setting value, construct a reference coordinate system through the pre-pollination point and the pollination path planning end point.

[0036] Based on the stamen orientation information of each style to be pollinated, determining the pollination path planning endpoint and pre-pollination point, specifically comprising: pre-constructing a virtual cylindrical structure corresponding to each style to be pollinated, wherein the virtual cylindrical structure includes a style radius; determining a designated bottom surface corresponding to the stamen orientation information in the virtual cylindrical structure according to the stamen orientation information of each style to be pollinated, and using the center of the designated bottom surface as the pollination path planning endpoint; taking the pollination path planning endpoint as the starting point and the pre-acquired initial pollination starting point as the end point, sequentially generating a plurality of reference pre-pollination points in a designated direction at intervals of a designated distance, wherein the designated direction is related to the stamen orientation information of each style to be pollinated; sequentially determining leaf obstacle information between the plurality of reference pre-pollination points and the predetermined pollination starting point, wherein the leaf obstacle information includes the number of leaf obstacles; and determining a pre-pollination point that meets the requirements from the plurality of reference pre-pollination points based on the leaf obstacle information, wherein the leaf obstacle information corresponding to the pre-pollination point that meets the requirements is that the number of leaf obstacles is less than a preset number threshold.

[0037] In one embodiment of the present specification, a virtual cylindrical structure corresponding to each style to be pollinated is constructed, and the entire stamen is approximately regarded as a cylinder with a radius of r and a height of h. According to the orientation of the stamen of each style to be pollinated, a designated bottom surface corresponding to the orientation of the stamen is determined in the corresponding virtual cylindrical structure, and the center of the designated bottom surface is used as the end point of the pollination path planning. For example, when the stamen is facing upward, the center of the upper circular surface of the cylinder corresponding to the stamen is used as the end point of the path planning, and the determined end point of the path planning is used as the starting point. A reference pre-pollination point is generated vertically at intervals until the distance threshold d0 is reached and the process stops, generating at least one reference pre-pollination point; that is, the designated direction is related to the orientation of the stamen. The distance threshold here refers to the distance between the first reference pre-pollination point and the last pre-pollination point, which is determined according to the leaf distribution of the current plant to be pollinated. The greater the leaf density and the further back the leaves are, the greater the distance threshold.

[0038] The robot arm's starting point and the generated reference pre-pollination point are sequentially determined to determine whether there are any obstacles, typically leaf obstacles. If there are no obstacles or only a few obstacles, this point is selected as the pre-pollination point. A few obstacles can be fewer than five leaf obstacles. If there are multiple obstacles, meaning at least five leaf obstacles, the next reference pre-pollination point is evaluated to determine the final pre-pollination point. If a suitable pre-pollination point has not been found by the time the distance threshold d0 is reached, the pollination machine is controlled to rotate slightly by θ, after which a new pre-pollination point is selected. In special cases, such as crop flowers with downward-facing openings close to the ground, the pre-pollination point is selected from the reference pre-pollination points generated by the above method, with the point below the lowest leaf being the criteria. The final pre-pollination point is then determined by selecting any point within the plane containing the lowest leaf. After the pre-pollination point is determined, the distance L between the pre-pollination point and the planned endpoint is calculated.

[0039] A reference coordinate system is constructed through the pre-pollination point and the pollination path planning endpoint, specifically including: connecting the pre-pollination point and the pollination path planning endpoint as the X-axis, and setting the positive direction of the X-axis as the direction of the pollination path planning endpoint specified by the pre-pollination point; based on the X-axis, determining a first plane containing the X-axis and perpendicular to the ground; determining the midpoint of the line connecting the pre-pollination point and the pollination path planning endpoint in the first plane, and constructing a Y-axis based on the midpoint, and the positive direction of the Y-axis is the growth direction of the plant to be pollinated, wherein the Y-axis belongs to the first plane and is perpendicular to the X-axis; based on the X-axis and the Y-axis, determining the Z-axis, and setting the positive direction of the Z-axis as the designated direction.

[0040] In one embodiment of the present specification, the distance L between the pre-pollination point and the end point of the path planning is determined relative to the distance setting value α. The distance setting value here can be set according to experimental data. If L is not greater than α, it means that the distance between the pre-pollination point and the end point of the path planning is small, and the improved BI-RRT algorithm based on the artificial potential field method can be used for path planning. If L is greater than α, it means that the distance between the pre-pollination point and the end point of the path planning is large, and the improved BI-RRT algorithm based on local optional points and artificial potential field method is used for path planning, which is to add preprocessing on the basis of the improved BI-RRT algorithm based on the artificial potential field method. The preprocessing here includes constructing a reference coordinate system through the pre-pollination point and the end point of the pollination path planning.

[0041] In one embodiment of this specification, Figure 2 A schematic diagram of a reference coordinate system provided in an embodiment of this specification, such as Figure 2As shown, connect the pre-pollination point and the end point of the path planning, and use this line as the X-axis of the coordinate system. The positive direction is the direction from the pre-pollination point to the end point of the path planning. Determine a plane perpendicular to the ground and with an X-axis. Find the midpoint of the straight line connecting the pre-pollination point and the end point of the path planning on this plane. Establish the Y-axis of the system at the midpoint. The Y-axis is perpendicular to the X-axis and on this plane. The positive direction of the Y-axis is related to the growth direction of the plant, and can also be related to the direction of the stamen. Figure 2 In the figure, if the pistil is facing the upper left, set the positive direction of the Y axis to the upward part, and establish the Z axis of the system with the positive direction pointing to the outside of the plant to construct a reference coordinate system.

[0042] Step S104: In the reference coordinate system, determine the set of leaf obstacles in the designated area corresponding to the pre-pollination point and the end point of the pollination path planning, and establish a sampling space based on the set of leaf obstacles in the designated area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing the pollination of the plant to be pollinated.

[0043] In the reference coordinate system, the leaf obstacle set within the designated area corresponding to the pre-pollination point and the pollination path planning end point is determined, specifically including: obtaining a pre-constructed virtual cylindrical structure corresponding to each style to be pollinated, determining the style radius in the virtual cylindrical structure, and generating a search radius based on the style radius; using the line between the pre-pollination point and the pollination path planning end point as the axis, generating a cylindrical leaf obstacle search area according to the search radius; within the cylindrical leaf obstacle search area, performing leaf obstacle search in a preset order to generate an ordered leaf obstacle set.

[0044] In one embodiment of the present specification, a search radius k1r is generated based on the style radius r in the virtual cylindrical structure. k1 can be set as required. The straight line connecting the pre-pollination point and the pollination path planning end point is used as the axis and k1r is used as the radius to generate a cylindrical leaf obstacle search area. Obstacles obsin (obsin0, obsin1…obsin) are searched in the cylindrical leaf obstacle search area. i ), according to the distance between each leaf obstacle and the pre-pollination point, in order from near to far, import them into the leaf obstacle set Z, find the obstacle obsin0 closest to the starting point and import it into the set Z, then find the obstacle obsin1 closest to the obstacle obsin0 and import it into the set Z, until obsin is imported into the set Z in order. i Import it into the set Z to obtain the ordered leaf obstacle set Z.

[0045] A sampling space is established based on the set of leaf obstacles in the designated area, specifically including: in the ordered set of leaf obstacles, coordinate analysis is performed on each leaf obstacle in turn based on the reference coordinate system to obtain multiple leaf point position coordinates corresponding to each leaf obstacle; based on the multiple leaf point position coordinates, multiple local optional points are determined from the multiple leaf points of each leaf obstacle, wherein the local optional points include any one or more of the leaf point corresponding to the maximum Z value, the leaf point corresponding to the minimum Z value, the leaf point corresponding to the maximum Y value, and the leaf point corresponding to the minimum Y value; the distance between each of the local optional points and the pre-pollination point or the planned end point of the pollination path is determined, and an ordered subset of local optional points corresponding to each leaf obstacle is generated in order of distance from small to large; the sampling space is established through the ordered subset of local optional points corresponding to each leaf obstacle.

[0046] In one embodiment of the present specification, in an ordered set of blade obstacles, a reference coordinate system is used to perform coordinate analysis on each blade obstacle in turn to obtain the coordinates of multiple blade points corresponding to each blade obstacle. n Find the points Zmax and Zmin with the maximum and minimum Z values ​​on (n∈[0,i]), as well as the two points Ymax and Ymin with the maximum and minimum Y axis values, and record them as local optional points. Figure 3 A schematic diagram of generating a local line segment is provided in the embodiment of this specification, such as Figure 3 As shown, the local optional points include Zmax, Zmin, Ymax, and Ymin. The same obstacle corresponds to Zmax, Zmin and Ymax, Ymin. Due to the special shape of the leaves, the distance between two local optional points is very close in special cases. The distance threshold can be set. When the distance is less than the distance threshold, the two local optional points with very close distances are regarded as one local optional point. Therefore, in special cases, there are only three local optional points in the end. Determine the distance between each local optional point and the pre-pollination point or the end point of pollination path planning. This is determined according to the selection rules of local optional points. The selection rules for local optional points are as follows: when there is a pre-pollination point on one side of the Y-axis, calculate the distance between the local optional point and the pre-pollination point. On the contrary, when there is a path planning end point on the Y-axis, calculate the distance between the local optional point and the path planning end point. In order from near to far, that is, the order of distance from small to large, the set Q is obtained. obsin0 , Q obsin1 …Q obsinn , that is, generate an ordered subset of local optional points corresponding to each leaf obstacle. The sampling space is established through the ordered subset of local optional points corresponding to each leaf obstacle.

[0047] The sampling space is established through an ordered subset of local optional points corresponding to each leaf obstacle, specifically including: determining at least one local optional point, at least one local line segment between the pre-pollination point and the pollination path planning end point based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point and the pollination path planning end point; generating a local cylinder corresponding to each local line segment to obtain multiple local cylinders; determining the overlapping area between the multiple local cylinders, and superimposing the other areas of the multiple local cylinders except the overlapping area to obtain the sampling space.

[0048] Based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point and the planned end point of the pollination path, at least one local optional point and at least one local line segment between the pre-pollination point and the planned end point of the pollination path are determined, specifically including: in the ordered set of leaf obstacles, determining the first leaf obstacle at the first position to determine the first ordered subset of local optional points corresponding to the first leaf obstacle; in the ordered subset of the first local optional points, determining the first local optional point at the first position, performing a connection judgment on the first local optional point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the first local optional point and the pre-pollination point; if the leaf obstacle exists between the line, obtaining a second local optional point in the ordered subset of the first local optional points that is located after the first local optional point; performing a connection judgment on the second local optional point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the second local optional point and the pre-pollination point; if the leaf obstacle does not exist between the lines, performing a connection judgment on the second local optional point and the end point of the pollination path planning to determine whether there is a leaf obstacle between the second local optional point and the end point of the pollination path planning; when the leaf obstacle does not exist between the lines, generating a first local line segment between the pre-pollination point and the second local optional point, and generating a second local line segment between the second local optional point and the end point of the pollination path planning.

[0049] After determining whether there is a leaf obstacle between the second local optional point and the end point of the pollination path planning, the method also includes: when the leaf obstacle exists between the connecting line, determining the second leaf obstacle located after the first leaf obstacle in the ordered leaf obstacle set to determine the ordered subset of second local optional points corresponding to the second leaf obstacle; performing a connection judgment between the designated local optional point located first in the ordered subset of second local optional points and the second local optional point to determine whether there is a leaf obstacle between the connecting line between the designated local optional point and the second local optional point; if there is no leaf obstacle between the connecting line between the designated local optional point and the second local optional point, determining whether there is a leaf obstacle between the designated local optional point and the end point of the pollination path planning; when the leaf obstacle does not exist between the connecting lines, generating a first local line segment between the pre-pollination point and the second local optional point, generating a second designated local line segment between the second local optional point and the designated local optional point, and generating a third local line segment between the designated local optional point and the end point of the pollination path planning.

[0050] In one embodiment of the present specification, Q is derived in positive order. obsin0 The first local optional point in is connected to the pre-pollination point to determine whether the line connecting the two is blocked by the obstacle leaf. If blocked, derive Q in positive order. obsin0 The second local optional point in the path is selected, and the connection between the second local optional point and the pre-pollination point is judged. If there is no obstruction between the second local optional point and the pre-pollination point, whether the connection line between the current second local optional point and the path planning end point is blocked by a leaf obstacle is judged. If there is no obstruction, the current second local optional point and the path planning end point are directly connected. The preprocessing is completed, and a local line segment between the pre-pollination point and the first local optional point, a local line segment between the first local optional point and the second local optional point, and a local line segment between the second local optional point and the path planning end point are generated.

[0051] If the line between the current second local optional point and the path planning end point is blocked by a leaf obstacle, continue to derive Q obsin1 The first local optional point in the obstacle is judged and connected with the local optional point selected on the previous obstacle obsin0, and the connection is carried out in sequence until it is connected to the last obstacle obsin i Or the end point ends preprocessing to obtain multiple local line segments.

[0052] In one embodiment of this specification, after the preprocessing is completed, multiple local line segments are obtained. A sampling space of BI-RRT using the improved artificial potential field method is established in the line segments generated by the preprocessing. The two ends of each of the multiple local line segments are extended respectively, and the extension distance is set according to requirements and can be set to a relatively small value. With the extended local line segment as the axis and k2r as the radius, a local cylinder corresponding to each extended local line segment is generated. Here, r is the radius of the flower column and k2 < k1, and k2 can be set according to requirements. The overlapping areas between the multiple local cylinders are determined, and the areas of the multiple local cylinders other than the overlapping areas are superimposed to obtain the sampling space.

[0053] In one embodiment of this specification, the improved BI-RRT algorithm based on the artificial potential field method is used for path planning. Four trees are grown between the starting point, the pre-pollination point, and the ending point of the pollination robot manipulator. The generation of new nodes is jointly determined by the gravitational force and the repulsive force in the artificial potential field method. The influence range of the repulsive force ρ0 is set. The larger ρ0 is, the larger the influence range of the repulsive force is, and the repulsive force will only play a role within this range. In addition, a probability threshold P0 is set, which can be obtained according to experimental data. When the node is within the influence range of the repulsive force, when the generated random probability is less than the probability threshold, the generation of the new node is determined by the random point; when the generated random probability is greater than the probability threshold, the generation of the new node is jointly determined by the gravitational force, the repulsive force, and the random point. A distance threshold d1 is set. When the distance between several continuously generated points is less than d1, the probability threshold is increased. When the node is outside the influence range of the repulsive force, when the generated random probability is less than the probability threshold, the generation of the new node is determined by the random point, and when the generated random probability is greater than the probability threshold, the generation of the new node is jointly determined by the gravitational force and the random point to generate multiple nodes for path planning and achieve path planning. It should be noted that the gravitational force of the Ta tree extends towards the ending point of the path planning, and the gravitational force of the Tb tree extends towards the Ta tree.

[0054] Through the above technical solution, the position of the style is obtained from the point cloud data of the plant to be pollinated, the current state of the style is analyzed to obtain the direction of the stamen, and the pollination path planning end point and the pre-pollination point are determined according to the direction of the stamen. That is, the starting point and end point in the pollination path are combined with the real-time state of the current plant to be pollinated, ensuring the accuracy and pertinence of the path planning starting point and end point, and can adapt to the postures of multiple plants; when the distance between the pre-pollination point and the pollination path planning end point is greater than the preset distance setting value, the pre-pollination point and the pollination path planning end point are combined to construct a reference coordinate system, and in the reference coordinate system, the leaf is The sampling space is obtained by analyzing leaf obstacles in combination with the leaf distribution of the plant, ensuring the pertinence of the analysis results. In addition, the sampling space is obtained in combination with the leaf distribution of the plant and the direction of the stamens, ensuring the preparation work before path planning and providing a sampling space without leaf obstacles. By improving the BI-RRT algorithm of the artificial potential field method, the pollination path corresponding to each style to be pollinated is generated in the established sampling space, which can solve the problem of too strong randomness of the RRT algorithm, improve the efficiency of path planning, and meet the actual pollination needs.

[0055] The embodiment of this specification also provides a path planning device for flower pollination, such as Figure 4 As shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0056] The point cloud data of the plants to be pollinated are collected to perform semantic segmentation on the point cloud data of the plants to be pollinated, and the style position data of the multiple styles to be pollinated of the plants to be pollinated are generated; the style posture data of each style to be pollinated are determined according to the style position data of each style to be pollinated by the point pair feature method, and the stamen orientation information of each style to be pollinated is determined based on the style posture data; based on the stamen orientation information of each style to be pollinated, the pollination path planning end point and the pre-pollination point are determined, and the distance between the pre-pollination point and the pollination path is calculated. The method comprises the following steps: determining the distance between the pre-pollination point and the pollination path planning end point; determining the distance between the pre-pollination point and the pollination path planning end point; and determining the leaf obstacle set within the specified area corresponding to the pre-pollination point and the pollination path planning end point. The method further comprises the following steps: establishing a sampling space based on the leaf obstacle set within the specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

[0057] The embodiments of this specification also provide a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0058] The point cloud data of the plants to be pollinated are collected to perform semantic segmentation on the point cloud data of the plants to be pollinated, and the style position data of the multiple styles to be pollinated of the plants to be pollinated are generated; the style posture data of each style to be pollinated are determined according to the style position data of each style to be pollinated by the point pair feature method, and the stamen orientation information of each style to be pollinated is determined based on the style posture data; based on the stamen orientation information of each style to be pollinated, the pollination path planning end point and the pre-pollination point are determined, and the distance between the pre-pollination point and the pollination path is calculated. The method comprises the following steps: determining the distance between the pre-pollination point and the pollination path planning end point; determining the distance between the pre-pollination point and the pollination path planning end point; and determining the leaf obstacle set within the specified area corresponding to the pre-pollination point and the pollination path planning end point. The method further comprises the following steps: establishing a sampling space based on the leaf obstacle set within the specified area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

[0059] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the device, apparatus, and non-volatile computer storage medium embodiments are generally similar to the method embodiments, so their descriptions are relatively simplified. For relevant details, refer to the descriptions of the method embodiments.

[0060] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] The devices and media provided in the embodiments of this specification correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0062] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0063] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0064] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0066] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0067] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0068] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0069] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0070] The foregoing description is merely one or more embodiments of this specification and is not intended to limit this specification. It will be apparent to those skilled in the art that various modifications and variations may be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of one or more embodiments of this specification are intended to be within the scope of the claims of this specification.

Claims

1. A path planning method for flower pollination, characterized in that: The method comprises: Collecting plant point cloud data of the plant to be pollinated, performing semantic segmentation on the plant point cloud data to generate style position data of a plurality of styles to be pollinated of the plant to be pollinated; Determining the style position data of each style to be pollinated by a point pair feature method according to the style position data of each style to be pollinated, and determining the stamen orientation information of each style to be pollinated based on the style position data; Based on the stamen orientation information of each style to be pollinated, determining the pollination path planning end point and the pre-pollination point, calculating the distance between the pre-pollination point and the pollination path planning end point, and when the distance is greater than a preset distance setting value, constructing a reference coordinate system through the pre-pollination point and the pollination path planning end point; Based on the stamen orientation information of each style to be pollinated, determining the pollination path planning end point and the pre-pollination point specifically includes: Pre-constructing a virtual cylindrical structure corresponding to each style to be pollinated, wherein the virtual cylindrical structure includes a style radius; According to the pistil orientation information of each style to be pollinated, a designated bottom surface corresponding to the pistil orientation information is determined in the virtual cylindrical structure, and the center of the designated bottom surface is used as the end point of the pollination path planning; Taking the planned pollination path endpoint as the starting point and the pre-acquired initial pollination starting point as the end point, a plurality of reference pre-pollination points are sequentially generated in a specified direction at intervals of a specified distance, wherein the specified direction is related to the stamen orientation information of each style to be pollinated; sequentially determining leaf obstacle information between the plurality of reference pre-pollination points and a predetermined pollination starting point, wherein the leaf obstacle information includes the number of leaf obstacles; Based on the leaf obstacle information, determining a pre-pollination point that meets the requirements among the multiple reference pre-pollination points, wherein the leaf obstacle information corresponding to the pre-pollination point that meets the requirements is that the number of leaf obstacles is less than a preset number threshold; In the reference coordinate system, the leaf obstacle set within the designated area corresponding to the pre-pollination point and the pollination path planning end point is determined, and a sampling space is established based on the leaf obstacle set within the designated area, so as to generate a pollination path corresponding to each style to be pollinated in the established sampling space through the BI-RRT algorithm of the improved artificial potential field method, thereby realizing pollination of the plant to be pollinated.

2. A path planning method for flower pollination according to claim 1, characterized in that: Constructing a reference coordinate system through the pre-pollination point and the pollination path planning end point specifically includes: Connecting the pre-pollination point and the pollination path planning endpoint as an X-axis, and setting the positive direction of the X-axis as the direction of the pollination path planning endpoint specified by the pre-pollination point; Based on the X-axis, determining a first plane that includes the X-axis and is perpendicular to the ground; Determining, in the first plane, a midpoint of a line connecting the pre-pollination point and the planned pollination path endpoint, and constructing a Y-axis based on the midpoint, wherein the positive direction of the Y-axis is the growth direction of the plant to be pollinated, wherein the Y-axis belongs to the first plane and is perpendicular to the X-axis; Based on the X-axis and the Y-axis, a Z-axis is determined, and a positive direction of the Z-axis is set as a designated direction.

3. A path planning method for flower pollination according to claim 1, characterized in that: Determining, in the reference coordinate system, a set of leaf obstacles within a designated area corresponding to the pre-pollination point and the pollination path planning endpoint specifically includes: Obtaining a pre-constructed virtual cylindrical structure corresponding to each style to be pollinated, determining the style radius in the virtual cylindrical structure, and generating a search radius based on the style radius; Using the line between the pre-pollination point and the planned pollination path endpoint as an axis, and according to the search radius, generating a cylindrical leaf obstacle search area; In the cylindrical blade obstacle search area, blade obstacle search is performed according to a preset order to generate an ordered blade obstacle set.

4. A path planning method for flower pollination according to claim 3, characterized in that: Establishing a sampling space based on the set of leaf obstacles within the designated area specifically includes: In the ordered blade obstacle set, coordinate analysis is performed on each blade obstacle in turn based on the reference coordinate system to obtain multiple blade point position coordinates corresponding to each blade obstacle; Determine a plurality of local selectable points from the plurality of blade points of each blade obstacle according to the plurality of blade point position coordinates, wherein the local selectable points include any one or more of the blade point corresponding to the maximum Z value, the blade point corresponding to the minimum Z value, the blade point corresponding to the maximum Y value, and the blade point corresponding to the minimum Y value; Determining the distance between each of the local selectable points and the pre-pollination point or the planned pollination path endpoint, and generating an ordered subset of local selectable points corresponding to each leaf obstacle in ascending order of distance; The sampling space is established by an ordered subset of local selectable points corresponding to each leaf obstacle.

5. A path planning method for flower pollination according to claim 4, characterized in that: The sampling space is established by an ordered subset of local selectable points corresponding to each blade obstacle, specifically including: Determine at least one local line segment between at least one local optional point, the pre-pollination point, and the pollination path planning endpoint based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point, and the pollination path planning endpoint; generating a local cylinder corresponding to each of the local line segments to obtain a plurality of local cylinders; An overlapping area between the multiple partial cylinders is determined, and other areas of the multiple partial cylinders except the overlapping area are superimposed to obtain the sampling space.

6. A path planning method for flower pollination according to claim 5, characterized in that: Determining at least one local line segment between at least one local optional point, the pre-pollination point, and the planned pollination path endpoint based on the ordered subset of local optional points corresponding to each leaf obstacle, the pre-pollination point, and the planned pollination path endpoint specifically includes: Determine a first leaf obstacle in the ordered leaf obstacle set to determine a first local optional point ordered subset corresponding to the first leaf obstacle; Determine a first local selectable point in the ordered subset of the first local selectable points, and perform a connection check between the first local selectable point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the first local selectable point and the pre-pollination point; If the blade obstacle exists between the connecting lines, obtaining a second local selectable point in the ordered subset of the first local selectable points that is located after the first local selectable point; Performing a connection judgment on the second locally selectable point and the pre-pollination point to determine whether there is a leaf obstacle between the line connecting the second locally selectable point and the pre-pollination point; If there is no leaf obstacle between the connecting lines, performing a connection judgment on the second local optional point and the planned pollination path end point to determine whether there is a leaf obstacle between the second local optional point and the planned pollination path end point; When the leaf obstacle does not exist between the connecting lines, a first local line segment is generated between the pre-pollination point and the second local optional point, and a second local line segment is generated between the second local optional point and the pollination path planning end point.

7. A path planning method for flower pollination according to claim 6, characterized in that: After determining whether there is a leaf obstacle between the second local selectable point and the planned pollination path end point, the method further includes: When the leaf obstacle exists between the connecting lines, determining a second leaf obstacle located after the first leaf obstacle in the ordered leaf obstacle set to determine a second local optional point ordered subset corresponding to the second leaf obstacle; Performing a connection judgment on the designated local selectable point located first in the ordered subset of the second local selectable points and the second local selectable point to determine whether there is a blade obstacle between the line connecting the designated local selectable point and the second local selectable point; If there is no leaf obstacle between the designated local optional point and the second local optional point, determining whether there is a leaf obstacle between the designated local optional point and the pollination path planning end point; When there is no leaf obstacle between the connecting lines, a first local line segment is generated between the pre-pollination point and the second local optional point, a second designated local line segment is generated between the second local optional point and the designated local optional point, and a third local line segment is generated between the designated local optional point and the end point of the pollination path planning.

8. A path planning device for flower pollination, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: execute the method according to any one of claims 1 to 7.

9. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to execute the method according to any one of claims 1 to 7.

Images