Double-arm task planning method for picking robot with deep canopy adaptive to fruit distribution characteristics

Abstract

The invention provides a double-arm task planning method for a deep-canopy picking robot with self-adaptive fruit distribution characteristics, and the method comprises the steps: extracting a hand-eye combination region of interest, segmenting a left / right arm exclusive region and a double-arm connection region, carrying out the pre-distribution according to the minimum difference value of the number of fruits distributed by double arms, and calculating the picking execution time of the left arm and the right arm based on a pre-distribution result; in the process of calculating the picking execution time of the left / right arm, solving the shortest path of the left / right arm through a genetic algorithm, and taking the shortest path as the execution path length of sequential planning of the left and right arms; in the process of solving the shortest path of the left / right arm, introducing a strong constraint that a time difference value between a picking path sequence planning starting point and a corresponding dangerous fruit is greater than TM / N3; according to the picking execution time of the left arm and the right arm, the number of fruits of the left arm and the right arm is redistributed, and finally the shortest path for picking the redistributed fruits of the two arms is obtained. The problems that the utilization rate of a mechanical arm is low and the picking success rate is low in the fruit picking process with large distribution position difference of deep-canopy fruit trees are solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent agricultural equipment, and in particular to a dual-arm task planning method for a picking robot with deep canopy adaptive fruit distribution characteristics. Background Art

[0002] The picking efficiency of the picking robot has not increased exponentially with the increase in the number of robotic arms. The dual-arm task planning of the dual-arm picking robot is the key to unleashing the potential of the dual arms. Compared with single-arm robots, dual-arm robots require more complex task planning strategies to achieve better collaboration and efficiency. However, existing research is difficult to meet the requirements of high-speed operation of picking robots in orchards. The main problems are as follows:

[0003] (1) In the actual environment of deep canopy orchards, the distribution uniformity of fruits is poor and the distribution depth varies greatly. The existing multi-arm task allocation methods either isolate the workspace to avoid the multi-arm operation area or allocate tasks based on a fixed workspace. These methods have poor adaptability to the orchard environment, resulting in low efficiency of the multi-arm picking robot.

[0004] (2) In the actual environment of deep canopy orchards, fruits are densely clustered. The existing planning methods either restrict the trajectory of the two arms or gradually select the coordinates farthest from other fruits for sorting according to the spatial distance of the fruit coordinates. This is inefficient and complex, resulting in a low picking success rate for the picking robot. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention provides a dual-arm task planning method for a harvesting robot with adaptive deep canopy fruit distribution characteristics, thereby achieving collision-free and parallel high-speed harvesting of fruits with large differences in deep canopy distribution by both arms.

[0006] The present invention achieves the above technical objectives through the following technical means.

[0007] A dual-arm task planning method for a harvesting robot based on deep canopy adaptive fruit distribution characteristics:

[0008] By combining the hand-eye combination of the fruit center point with the coordinate set of interest [X, Y, Z] all The fruit coordinates in the left arm are judged and the fruit set in the left arm exclusive domain is divided into [x, y, z] LR , Fruit set in the right arm exclusive domain [x,y,z] RR and the fruit set in the intersection of the two arms [x,y,z] MR , for the fruit coordinate set of the double-arm intersection area [x, y, z] MR The fruit coordinates in the fruit are sorted according to the size of the y value and divided into [x, y, z] MR1 and [x,y,z] MR2 Two regions;

[0009] Based on the parity of the number of fruits in the harvesting space, the hand-eye combination of interest in the fruit center point coordinate set [X, Y, Z] all The number of fruits in the quartile is pre-allocated based on the minimum difference in quantity:

[0010] If N 1 +N 2 +N 3 is an even number, then:

[0011] If N 1 +N 2 +N 3 is an odd number, then: or

[0012] Among them, N 1 is the fruit set in the left arm exclusive domain [x,y,z] LR The number of fruits in the 2 is the fruit set in the exclusive domain of the right arm [x,y,z] RR The number of fruits in the 3 is the fruit set [x,y,z] in the intersection of the two arms MR The number of fruits in the 4 is [x,y,z] MR1 The number of fruits in the 5 is [x,y,z] MR2 The number of fruits in

[0013] According to the pre-allocation results, calculate the left arm picking execution time T L and the right arm picking execution time T R , in finding T L and T R When the left arm sequential planning starting point is set to the initial pose coordinates (x l ,y l ,z l ), set the starting point of the right arm sequential planning to the initial pose coordinates of the left robot end effector (x r ,y r ,z r ), the shortest path of the left and right arms is obtained by genetic algorithm; in the process of obtaining the shortest path of the left and right arms, the time difference between the starting point and the dangerous fruit pair is greater than T when the picking path sequence planning is introduced M / N 3 A strong constraint; where T M The time taken by the robotic arm to pick all the fruits in the handover area;

[0014] Compare the left arm picking execution time T L and the right arm picking execution time TR The size of redistribution of fruits: if T L Greater than T R , then the left arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result LR and the fruit coordinate set [x,y,z] MR1 Inside fruits are allocated to the right arm, and [x,y,z] is redistributed first LR and [x,y,z] MR1 The fruit with the largest y coordinate; if T L Less than T R , then the right arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result RR and [x,y,z] MR2 Inside fruits are allocated to the left arm, and [x,y,z] is redistributed first RR and [x,y,z] MR2 The fruit with the smallest y coordinate, where ΔT is the difference in the execution time of picking by the left and right arms;

[0015] According to the redistribution results, the shortest path of the redistributed fruits after two-arm picking is obtained through genetic algorithm.

[0016] Furthermore, the fruit center point hand-eye combination of interest coordinate set [X, Y, Z] all Obtained through:

[0017] Convert all center point coordinates obtained by the depth cameras installed on the two six-degree-of-freedom robotic arms to the robot body global coordinate system U to generate the fruit center point camera recognition coordinate set [x, y, z] camera , remove the repeated positioning fruits in the field of view images of the two depth cameras, and further generate the camera non-repeated identification coordinate set [x, y, z] of the fruit center point nd , remove the redundant fruit center coordinates outside the robot arm workspace, and further generate the fruit center hand-eye combination of interest coordinate set [X, Y, Z] all .

[0018] Furthermore, the hand-eye combination of the fruit center point is interested in the coordinate set [X, Y, Z] all The fruit coordinates in the left arm are judged and the fruit set in the left arm exclusive domain is divided into [x, y, z] LR , Fruit set in the right arm exclusive domain [x,y,z] RR and the fruit set in the intersection of the two arms [x,y,z] MR , specifically:

[0019] The fruit coordinates satisfying formula (1) are taken as the fruit set [x, y, z] in the exclusive domain of the left arm.LR :

[0020]

[0021] The fruit coordinates satisfying formula (2) are taken as the fruit set [x, y, z] in the exclusive domain of the right arm. RR :

[0022]

[0023] The fruit coordinates satisfying formula (3) are taken as the fruit set [x, y, z] in the intersection domain of the two arms. MR :

[0024]

[0025] Among them, (x left ,y left ,z left ) is the base coordinate of the left manipulator, (x right ,y right ,z right ) is the base coordinate of the right manipulator, R robot is the working radius of the robot arm.

[0026] Furthermore, the time difference between the introduction of dangerous fruits and the arrival of the corresponding fruit picking path sequence planning starting point is greater than T M / N 3 The strong constraint is as follows: before each generation of crossover mutation operation, determine the difference between the time point when the left arm moves from the initial posture to the dangerous fruit on the left and the time point when the right arm moves from the initial posture to the dangerous fruit on the right, which will be greater than the time difference and less than or equal to T M / N 3 The corresponding path is removed, and the path that meets the time limit is obtained iteratively. The path with the smallest value among the paths that meet the time limit is selected as the shortest path.

[0027] Furthermore, the dangerous fruit pair consisting of the left dangerous fruit and the right dangerous fruit is formed by the collision area V D The relative positions of the fruit coordinates assigned to different robotic arms in the space are judged, and all fruit coordinate pairs that satisfy the following formula are regarded as dangerous fruit pairs:

[0028]

[0029] Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) is the collision area V D Inner fruit coordinates, R mis the maximum working radius of the end effector.

[0030] Furthermore, the collision area V D Determine it in the following way: Extract the fruit coordinates [x, y, z] allocated to the left arm in the pre-allocation result LR and [x,y,z] MR1 The coordinate with the largest y value (x 1 ,y 1 ,z 1 ), the fruit coordinates assigned to the right arm [x, y, z] RR and [x,y,z] MR2 The coordinate with the smallest y value (x 2 ,y 2 ,z 2 ), the left and right arm collision area V D Let the y coordinate be space area.

[0031] Furthermore, the time T taken by the robotic arm to pick all the fruits in the handover area is M Calculated by:

[0032]

[0033] Among them, S is the execution path length of the sequential planning of the handover domain, n is the number of fruits in the handover domain of the robot arm, K is the distance between the end of the robot arm and the x coordinate of the fruit, V m is the linear moving speed of the end of the robot arm, T d is the opening or closing time of the end effector.

[0034] Furthermore, the left arm picking execution time T is calculated L and the right arm picking execution time T R When , the shortest paths of the left and right arms obtained by the genetic algorithm are used as the execution path lengths of the left and right arm sequential planning respectively, and the number of fruits is the number of left and right arm allocations in the pre-allocation results.

[0035] The beneficial effects of the present invention are as follows: the present invention realizes fast redundant fruit coordinate elimination by extracting the hand-eye combination interest region, divides the fruits in the left / right arm exclusive domain and the double-arm intersection domain based on the hand-eye combination interest region, realizes pre-allocation with the minimum difference in the number of fruits allocated to the double arms, and calculates the left and right arm picking execution time based on the pre-allocation result; in the process of calculating the left and right arm picking execution time, the shortest path of the left and right arms is obtained by genetic algorithm, and is used as the execution path length of the left and right arm sequential planning; in the process of obtaining the shortest path of the left and right arms, the time difference between the starting point of the picking path sequential planning and the corresponding dangerous fruit pair is introduced to be greater than T M / N3 The invention solves the problems of low utilization rate of the robot arm and low success rate of picking during the picking operation of the dual-arm robot. The method is simple, reliable and highly practical. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of the harvesting robot according to the present invention;

[0037] Figure 2 This is a top view of the harvesting robot of the present invention;

[0038] Figure 3 This is a coordinate system diagram of the robot body of the present invention;

[0039] Figure 4 This is a schematic diagram of the single-arm exclusive domain and the double-arm junction domain of the present invention;

[0040] Figure 5 This is a schematic diagram of the distance between dangerous fruits according to the present invention;

[0041] Figure 6 This is a flow chart of the dual-arm task planning method for the picking robot of the present invention;

[0042] In the figure: 1. End effector A, 2. Depth camera A, 3. End effector B, 4. Depth camera B, 5. Deep canopy fruit tree, 6. Right arm robotic arm joint, 7. Left arm robotic arm joint. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0044] The present invention provides a deep canopy adaptive fruit distribution feature picking robot dual-arm task planning method, including a hand-eye combined region of interest extraction sub-method, a dual-arm task pre-allocation sub-method and a time-optimal task planning sub-method, see Figure 6 .

[0045] like Figure 2 As shown in the figure, in order to achieve full coverage of deep canopy harvesting, two six-degree-of-freedom robotic arms are installed on the robot close to the fruit tree side and at a distance of L from the edge of the robot body in the depth direction. robot Where, L robot is the radius of the flange plate of the robot arm installation base; in the forward direction, the two six-degree-of-freedom robot arms are installed on the upper part of the robot body and are R robot Where, R robot is the working radius of the six-degree-of-freedom robot. On this basis, Figure 3 As shown, the robot body global coordinate system U is established with the depth direction as the X-axis, the forward direction as the Y-axis, the vertical upward direction as the Z-axis, and the center point of the two robot arm bases as the origin O. In this embodiment, R robot =860mm, L robot =70mm.

[0046] In the configuration of eyes on hands, the depth camera B4 and the end effector B3 are installed on the left arm, and the depth camera A2 and the end effector A1 are installed on the right arm. The deep canopy fruit tree 5 is identified and located by the depth camera B4 and the depth camera B2. For specific installation, see Figure 1 .

[0047] 1. Process of the hand-eye combined region of interest extraction sub-method

[0048] Convert all center point coordinates obtained by the two cameras to the robot body global coordinate system U to generate the fruit center point camera recognition coordinate set [x, y, z] camera In the robot body global coordinate system U, for fruits whose fruit center distance A is less than distance D (where D is the average diameter of the fruit), they are considered to be the same fruit, thereby eliminating the repeated positioning fruits in the field of view images of the two cameras and generating a non-repeated recognition coordinate set [x, y, z] of the fruit center point camera. nd .

[0049] For the fruit center point camera, there is no duplicate recognition coordinate set [x, y, z] nd , according to the robot arm workspace V r And the left robot base coordinate (x left ,y left ,z left )、right robot base coordinates (x right ,y right ,z right ), the fruits that do not satisfy formula (1) or (2) are regarded as redundant fruits outside the robot workspace:

[0050] (xx left ) 2 +(yy left ) 2 +(zz left ) 2 ≤R robot 2 (1)

[0051] (xx right ) 2 +(yy right ) 2 +(zz right ) 2≤R robot 2 (2)

[0052] Among them, (x, y, z) is the fruit coordinate, which comes from the non-repeated recognition coordinate set [x, y, z] of the camera at the center of the fruit nd .

[0053] Eliminate the redundant fruit center coordinates outside the robot arm workspace and generate the fruit center hand-eye combination of interest coordinate set [X, Y, Z] all .

[0054] 2. Process of the Dual-Arm Task Pre-allocation Sub-Method

[0055] like Figure 4 As shown, by combining the hand-eye combination of the fruit center point with the coordinate set of interest [X, Y, Z] all The fruit coordinates in the left arm are judged, and the fruit coordinates that satisfy formula (3) are taken as the fruit set [x, y, z] in the left arm exclusive domain. LR :

[0056]

[0057] The fruit coordinates satisfying formula (4) are taken as the fruit set [x, y, z] in the exclusive domain of the right arm. RR :

[0058]

[0059] The fruit coordinates satisfying formula (5) are taken as the fruit set [x, y, z] in the intersection domain of the two arms. MR :

[0060]

[0061] In this embodiment, the fruit center point hand-eye combination coordinate set of interest [X, Y, Z] all The values ​​are: [(0.563, -1.234, -0.622), (0.463, -1.034, 0.622), (0.563, -0.934, 0.322), (0.563, -0.534, 0.522), (0.663, -0.234, 0.732), (0.666, -0.244, 0. 732), (0.463, 0.255, 0.222), (0.363, 0.634, 0.423), (0.463, 0.734, 0.522), (0.763, 1.034, 0.622), (0.663, 1.334, 0.522), (0.563, 1.225, 0.786)].

[0062] According to formula (3), determine the fruit coordinate set [x, y, z] of the left arm exclusive domain LR The number of fruits inside is set to N 1 According to formula (4), the fruit coordinate set [x, y, z] of the right arm exclusive domain is determined RR The number of fruits inside is set to N 2 According to formula (5), the fruit coordinate set [x, y, z] of the double-arm junction area is determined MR The number of fruits inside is set to N 3 In this embodiment, N 1 is 6, N 2 is 6, N 3 is 3.

[0063] The coordinate set of the fruit in the intersection of the two arms [x, y, z] MR The fruit coordinates in the fruit are sorted according to the size of the y value and divided into [x, y, z] MR1 and [x,y,z] MR2 Two regions, record N 4 is the coordinate set of the fruit on the left arm in the junction area [x, y, z] MR1 The number of fruit coordinates, N 5 is the coordinate set of the fruit in the right arm of the junction area [x, y, z] MR2 The number of fruit coordinates; based on the parity of the number of fruits in the harvesting space, the hand-eye combination of interest in the fruit center point coordinate set [X, Y, Z] all The number of fruits in the quartile is pre-allocated based on the minimum difference in quantity. The allocation method is as follows:

[0064] If N 1 +N 2 +N 3 is an even number, then:

[0065] If N 1 +N 2 +N 3 is an odd number, then: or

[0066] In this embodiment, the fruit coordinates assigned to the left arm are [(0.563, -1.234, -0.622), (0.463, -1.034, 0.622), (0.563, -0.934, 0.322), (0.563, -0.534, 0.522), (0.663, -0.234, 0.732), (0.666, -0.244, 0 .732)]; the fruit coordinates assigned to the left arm are [(0.463, 0.255, 0.222), (0.363, 0.634, 0.423), (0.463, 0.734, 0.522), (0.763, 1.034, 0.622), (0.663, 1.334, 0.522), (0.563, 1.225, 0.786)].

[0067] 3. Process of the Time Optimal Task Planning Sub-Method

[0068] For continuous picking of multiple fruits, the right arm joint 6 and the left arm joint 7 are kept external during the picking process. The end of the robot arm is first moved to the sequential planning point of the picking path (i.e., Kcm before the x-coordinate of the fruit), the end effector is opened, and it moves forward horizontally by Kcm. The end effector is closed to grab the fruit and moves back horizontally by Kcm. After the end effector is released to release the fruit, the end effector is closed again to maintain the end linear movement speed V m , move to the next execution point of the fruit picking path sequence planning, and repeat until all coordinates are completed. The time taken to pick all fruits is calculated as follows:

[0069]

[0070] Among them, S is the execution path length of sequential planning, n is the number of fruits, V m is the linear moving speed of the end of the robot arm, T d is the opening or closing time of the end effector; in this embodiment, V m is 0.2m / s, T d is 1s, K is 10cm.

[0071] For the fruit coordinate set of the double-arm intersection area [x, y, z] MR Inner fruit, set the starting point of sequential planning to the fruit coordinate set [x, y, z] of the intersection of the two arms MR The coordinate point with the smallest x coordinate (x s ,y s ,z s ), the shortest path is obtained by genetic algorithm as the execution path length of the handover domain sequence planning, and the time T used to pick all the fruits in the handover domain is calculated according to formula (6). M(In the process of obtaining, n is the number of fruits in the handover area of ​​the robot arm), and the average time taken is T M / N 3 In this embodiment, the average time used is 3.29 seconds.

[0072] According to the pre-allocation results, calculate the left arm picking execution time T L , right arm picking execution time T R , in finding T L and T R When the left arm sequential planning starting point is set to the initial pose coordinates (x l ,y l ,z l ), set the starting point of the right arm sequential planning to the initial pose coordinates of the left robot end effector (x r ,y r ,z r ), the shortest path of the left and right arms is obtained through genetic algorithm, and used as the execution path length of the left and right arm sequential planning, while the number of fruits is the result of pre-allocation.

[0073] In the process of finding the shortest path for the left and right arms, the picking path sequence planning is introduced. When the time difference between the starting point and the corresponding dangerous fruit pair is greater than T M / N 3 Specifically, before each generation of crossover mutation operation, determine the difference between the time point when the left arm moves from the initial posture to the dangerous fruit on the left and the time point when the right arm moves from the initial posture to the dangerous fruit on the right, which will be greater than the time difference and less than or equal to T M / N 3 The corresponding path is removed, and the path that meets the time limit is obtained iteratively. The path with the smallest value among the paths that meet the time limit is selected as the shortest path.

[0074] Dangerous fruit pairs are defined in the following way: D The relative positions of the fruit coordinates assigned to different robot arms in the space are judged, and all fruit coordinate pairs that satisfy formula (7) are regarded as dangerous fruit pairs (see Figure 5 ):

[0075]

[0076] Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) is the collision area V DInner fruit coordinates; in this embodiment, the dangerous fruit pair is [(0.663, -0.234, 0.732), (0.666, -0.244, 0.732)].

[0077] Collision area V D Determine it in the following way: Extract the fruit coordinates [x, y, z] assigned to the left arm in the task assignment result LR and [x,y,z] MR1 The coordinate with the largest y value (x 1 ,y 1 ,z 1 ), the fruit coordinates assigned to the right arm [x, y, z] RR and [x,y,z] MR2 The coordinate with the smallest y value (x 2 ,y 2 ,z 2 ), the left and right arm collision area V D Let the y coordinate be The spatial region, where R m is the maximum working radius of the end effector.

[0078] Compare the left arm picking execution time T L and the right arm picking execution time T R , reallocate, so that the robotic arm can quickly complete the picking task.

[0079] If the left arm picking execution time is T L Greater than the right arm picking execution time T R , then the left arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result LR and the coordinate set of the fruit on the left arm of the junction area [x,y,z] MR1 Inside fruits are allocated to the right arm, and [x,y,z] is redistributed first LR and [x,y,z] MR1 The fruit with the largest y coordinate; if the left arm picks the fruit, the execution time is T L Less than the right arm picking execution time T R , then the right arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result RR and the coordinate set of the fruit on the right arm in the intersection domain [x,y,z] MR2 Inside fruits are allocated to the left arm, and [x,y,z] is redistributed first RR and [x,y,z] MR2 The fruit with the smallest y coordinate is shown in the figure, where ΔT is the difference in execution time for picking by the left and right arms.

[0080] According to the redistribution results, the collision area is regenerated and the dangerous fruit pairs are detected. The shortest path for picking the redistributed fruits with two arms is obtained through a genetic algorithm.

[0081] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. A dual-arm task planning method for a deep canopy-adaptive fruit distribution picking robot, characterized in that: By combining the hand-eye combination of the fruit center point with the coordinate set of interest [X, Y, Z] all The fruit coordinates in the left arm are judged and the fruit set in the left arm exclusive domain is divided into [x, y, z] LR , Fruit set in the right arm exclusive domain [x,y,z] RR and the fruit set in the intersection of the two arms [x,y,z] MR , for the fruit coordinate set of the double-arm intersection area [x, y, z] MR The fruit coordinates in the fruit are sorted according to the size of the y value and divided into [x, y, z] MR1 and [x,y,z] MR2 Two regions; Based on the parity of the number of fruits in the harvesting space, the hand-eye combination of interest in the fruit center point coordinate set [X, Y, Z] all The number of fruits in the quartile is pre-allocated based on the minimum difference in quantity: If N1+N2+N3 is an even number, then: If N1+N2+N3 is an odd number, then: or Among them, N1 is the fruit set [x,y,z] in the exclusive domain of the left arm LR N2 is the number of fruits in the exclusive domain of the right arm [x, y, z] RR N3 is the number of fruits in the intersection of the two arms [x, y, z] MR The number of fruits in the, N4 is [x,y,z] MR1 The number of fruits in the, N5 is [x,y,z] MR2 The number of fruits in According to the pre-allocation results, calculate the left arm picking execution time T L and the right arm picking execution time T R , in finding T L and T R When the left arm sequential planning starting point is set to the initial pose coordinates (x l ,y l ,z l ), set the starting point of the right arm sequential planning to the initial pose coordinates of the left robot end effector (x r ,y r ,z r ), the shortest path of the left and right arms is obtained by genetic algorithm; in the process of obtaining the shortest path of the left and right arms, the time difference between the starting point and the dangerous fruit pair is greater than T when the picking path sequence planning is introduced M / N3 strong constraint; where T M The time taken by the robotic arm to pick all the fruits in the handover area; Compare the left arm picking execution time T L and the right arm picking execution time T R The size of redistribution of fruits: if T L Greater than T R , then the left arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result LR and the fruit coordinate set [x,y,z] MR1 Inside fruits are allocated to the right arm, and [x,y,z] is redistributed first LR and [x,y,z] MR1 The fruit with the largest y coordinate; if T L Less than T R , then the right arm exclusive domain fruit coordinate set [x, y, z] in the pre-allocated result RR and [x,y,z] MR2 Inside fruits are allocated to the left arm, and [x,y,z] is redistributed first RR and [x,y,z] MR2 The fruit with the smallest y coordinate, where ΔT is the difference in the execution time of picking by the left and right arms; According to the redistribution results, the shortest path of the redistributed fruits after two-arm picking is obtained through genetic algorithm.

2. The dual-arm task planning method for a harvesting robot according to claim 1, characterized in that: The fruit center point hand-eye combination of interest coordinate set [X, Y, Z] all Obtained through: Convert all center point coordinates obtained by the depth cameras installed on the two six-degree-of-freedom robotic arms to the robot body global coordinate system U to generate the fruit center point camera recognition coordinate set [x, y, z] camera , remove the repeated positioning fruits in the field of view images of the two depth cameras, and further generate the camera non-repeated identification coordinate set [x, y, z] of the fruit center point nd , remove the redundant fruit center coordinates outside the robot arm workspace, and further generate the fruit center hand-eye combination of interest coordinate set [X, Y, Z] all .

3. The picking robot dual-arm task planning method according to claim 1 is characterized in that: The coordinate set [X, Y, Z] of the hand-eye combination of the fruit center point is of interest all The fruit coordinates in the left arm are judged and the fruit set in the left arm exclusive domain is divided into [x, y, z] LR , Fruit set in the right arm exclusive domain [x,y,z] RR and the fruit set in the intersection of the two arms [x,y,z] MR , specifically: The fruit coordinates satisfying formula (1) are taken as the fruit set [x, y, z] in the exclusive domain of the left arm. LR : The fruit coordinates satisfying formula (2) are taken as the fruit set [x, y, z] in the exclusive domain of the right arm. RR : The fruit coordinates satisfying formula (3) are taken as the fruit set [x, y, z] in the intersection domain of the two arms. MR : Among them, (x left ,y left ,z left ) is the base coordinate of the left manipulator, (x right ,y right ,z right ) is the base coordinate of the right manipulator, R robot is the working radius of the robot arm.

4. The dual-arm task planning method for a harvesting robot according to claim 1, characterized in that: The time difference between the introduction of dangerous fruits and the arrival of the corresponding fruit picking path sequence planning starting point is greater than T M / N3 strong constraints, specifically: before each generation of crossover mutation operation, determine the time difference between the left arm's initial position to the left dangerous fruit and the right arm's initial position to the right dangerous fruit, which will be greater than the time difference and less than or equal to T M / The path corresponding to N3 is removed, and the path that meets the time limit is obtained by iteration. The path with the smallest value among the paths that meet the time limit is selected as the shortest path.

5. The dual-arm task planning method for a harvesting robot according to claim 4, characterized in that: The dangerous fruit pair consisting of the dangerous fruit on the left and the dangerous fruit on the right is formed by the collision area V D The relative positions of the fruit coordinates assigned to different robotic arms in the space are judged, and all fruit coordinate pairs that satisfy the following formula are regarded as dangerous fruit pairs: Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) is the collision area V D Inner fruit coordinates, R m is the maximum working radius of the end effector.

6. The picking robot dual-arm task planning method according to claim 5 is characterized in that: The collision area V D Determine it in the following way: Extract the fruit coordinates [x, y, z] allocated to the left arm in the pre-allocation result LR and [x,y,z] MR1 The coordinate with the largest y value (x1, y1, z1), the coordinate of the fruit assigned to the right arm [x, y, z] RR and [x,y,z] MR2 The coordinate with the smallest y value (x2, y2, z2) will collide with the left and right arms in the area V D Let the y coordinate be space area.

7. The picking robot dual-arm task planning method according to claim 1 is characterized in that: The time T taken by the robotic arm to pick all the fruits in the handover area M Calculated by: Among them, S is the execution path length of the sequential planning of the handover domain, n is the number of fruits in the handover domain of the robot arm, K is the distance between the end of the robot arm and the x coordinate of the fruit, V m is the linear moving speed of the end of the robot arm, T d is the opening or closing time of the end effector.

8. The dual-arm task planning method for a harvesting robot according to claim 7, characterized in that: Calculate the left arm picking execution time T L and the right arm picking execution time T R When , the shortest paths of the left and right arms obtained by the genetic algorithm are used as the execution path lengths of the left and right arm sequential planning respectively, and the number of fruits is the number of left and right arm allocations in the pre-allocation results.

Images