A deep crown layer self-adaptive fruit distribution characteristic picking robot dual-arm task planning method

Abstract

The application provides a deep crown layer self-adaptive fruit distribution feature picking robot dual-arm task planning method, extracts a hand-eye combined region of interest, segments left / right arm exclusive domains and dual-arm handover domains, performs pre-allocation according to a minimum difference value of fruit quantity allocated to the dual arms, calculates left / right arm picking execution time based on the pre-allocation result, calculates the shortest path of the left / right arm through a genetic algorithm in the process of calculating the left / right arm picking execution time, and uses the shortest path as an execution path length of left / right arm sequential planning; in the process of calculating the shortest path of the left / right arm, a strong constraint of a time difference value greater than T M / N3 from a starting point of picking path sequential planning to a corresponding dangerous fruit is introduced; the fruit quantity of the left / right arm is re-allocated according to the size of the left / right arm picking execution time, and the shortest path of the dual-arm picking after re-allocation is finally calculated. The application solves the problems of low mechanical arm utilization and low picking success rate in fruit picking of deep crown layer fruit trees with large distribution position differences.

Description

Technical Field

[0001] This invention relates to the field of intelligent agricultural equipment, and in particular to a dual-arm task planning method for a harvesting robot that adapts to the fruit distribution characteristics in deep canopy layers. Background Technology

[0002] The harvesting efficiency of picking robots does not increase exponentially with the increase in the number of robotic arms. Dual-arm task planning is key to unlocking the potential of dual-arm picking robots. Compared to single-arm robots, dual-arm robots require more complex task planning strategies to achieve better collaboration and efficiency. However, existing research falls short of meeting the requirements for high-speed operation of picking robots in orchards, mainly due to the following problems:

[0003] (1) In the actual environment of deep canopy orchards, the distribution uniformity of fruits is poor and the distribution depth varies greatly. Existing multi-arm task allocation methods either isolate the work space to avoid multi-arm operation areas or allocate tasks based on fixed work spaces. These methods are not adaptable to the orchard environment and result in low efficiency of multi-arm picking robots.

[0004] (2) In the actual environment of deep canopy orchards, fruits are densely clustered. Existing planning methods either restrict the planning trajectory of the two arms or sort the fruits by gradually selecting the coordinates of the furthest fruits from the other fruits according to the spatial distance of the fruit coordinates. This is inefficient and complex, resulting in a low success rate of picking by the picking robot. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a dual-arm task planning method for a harvesting robot that adapts to the fruit distribution characteristics in the deep canopy, enabling collision-free parallel high-speed harvesting of fruits with large differences in distribution within the deep canopy.

[0006] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0007] A dual-arm task planning method for a harvesting robot that adapts to fruit distribution characteristics in deep canopy:

[0008] By considering the coordinate set [X,Y,Z] of the center point of the fruit in a hand-eye combination, we can determine the coordinates of interest. all The coordinates of the fruits within the domain are used to determine the set of fruits [x, y, z] within the left arm's exclusive domain. LR The set of fruits within the right arm's exclusive domain [x,y,z] RR The set of fruits [x, y, z] within the area where the arms intersect. MR For the set of fruit coordinates [x,y,z] at the intersection of the two arms MR The fruit coordinates within the range are sorted according to their y-values ​​and divided into [x, y, z]. MR1 and [x,y,z] MR2 Two regions;

[0009] with the parity of the number of fruits in the collection space, the set of coordinates [X, Y, Z] of interest to the hand-eye combination of the center point of the fruit all The number of fruits in the inner fruit set [x, y, z] is pre-assigned according to the minimum difference in the number of fruits:

[0010] If N1+N2+N3 is even, then:

[0011] If N1+N2+N3 is odd, then: or

[0012] Where N1 is the number of fruits in the left arm exclusive domain fruit set [x, y, z] LR , N2 is the number of fruits in the right arm exclusive domain fruit set [x, y, z] RR , N3 is the number of fruits in the double arm interface domain fruit set [x, y, z] MR , N4 is the number of fruits in [x, y, z] MR1 , N5 is the number of fruits in [x, y, z] MR2 ;

[0013] According to the pre-assignment result, the left arm picking execution time T L and the right arm picking execution time T R are calculated, and in the calculation of T L and T R , the left arm sequential planning starting point is set as the initial pose coordinates (x l , y l , z l ) of the left arm end effector, and the right arm sequential planning starting point is set as the initial pose coordinates (x r , y r , z r ) of the left arm end effector, and the shortest path of the left and right arms is calculated by genetic algorithm; in the process of calculating the shortest path of the left and right arms, a strong constraint is introduced that the time difference between the arrival of the picking path sequential planning starting point to the dangerous fruit is greater than T M / N3; wherein T M is the time used by the arm to pick up all the fruits in the interface domain;

[0014] The size of the left arm picking execution time T L and the right arm picking execution time T R is compared, and the fruits are re-assigned: if T L is greater than T R , then the left arm exclusive domain fruit coordinate set [x, y, z] LR and the fruit coordinate set [x, y, z] MR1 in the pre-assignment result are re-assigned The fruit is assigned to the right arm, and the fruit coordinates [x, y, z] are preferentially reassigned LR and [x, y, z] MR1 The fruit with the largest y coordinate in [x, y, z] L T R If T RR The fruit coordinates [x, y, z] in the pre-assignment result are reassigned to the right arm exclusive domain MR2 and [x, y, z] RR The fruit with the smallest y coordinate in [x, y, z] MR2 , where ΔT is the difference between the left and right arm picking execution times; The fruit is assigned to the left arm, and the fruit coordinates [x, y, z] are preferentially reassigned RR and [x, y, z] MR2 The fruit with the smallest y coordinate in [x, y, z] MR2 , where ΔT is the difference between the left and right arm picking execution times;

[0015] According to the reassignment result, the shortest path of the fruit after reassignment by the dual-arm picking is obtained by a genetic algorithm.

[0016] Further, the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all is obtained by the following method:

[0017] All center point coordinates obtained by the depth cameras installed on the two six-degree-of-freedom mechanical arms are converted to the robot body global coordinate system U to generate the fruit center point camera recognition coordinate set [x, y, z] camera , and the fruit center point camera non-repeated recognition coordinate set [x, y, z] nd is further generated by eliminating redundant fruit center point coordinates outside the mechanical arm workspace, and the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all is further generated.

[0018] Further, the fruit coordinates in the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all are judged to divide the left arm exclusive domain fruit set [x, y, z] LR , the right arm exclusive domain fruit set [x, y, z] RR , and the dual-arm interface domain fruit set [x, y, z] MR , specifically:

[0019] The fruit coordinates satisfying formula (1) are taken as the left arm exclusive domain fruit set [x, y, z] LR :

[0020]

[0021] The fruit coordinates satisfying formula (2) are taken as the right arm exclusive domain fruit set [x, y, z] RR :

[0022]

[0023] The fruit coordinates satisfying formula (3) are taken as the fruit set [x, y, z] in the double-arm handover domain MR :

[0024]

[0025] where (x left , y left , z left ) is the left robot arm base coordinate, (x right , y right , z right ) is the right robot arm base coordinate, and R robot is the robot arm working radius.

[0026] Further, the introduction of the dangerous fruit reaches the corresponding fruit picking path order planning starting point time difference value greater than T M / N3 strong constraint, specifically: before each generation of crossover and mutation operation, determine the difference between the time point when the left arm moves from the initial pose to the left dangerous fruit and the time point when the right arm moves from the initial pose to the right dangerous fruit, which will be greater than the time difference value less than or equal to T M / N3 corresponding path is removed, and the path satisfying the time limit is obtained by iteration, and the smallest value in the path satisfying the time limit is selected as the shortest path.

[0027] Still further, the dangerous fruit pair composed of the left dangerous fruit and the right dangerous fruit is determined by judging the relative position of the fruit coordinates allocated to different robot arms in the collision region V D , and all fruit coordinate pairs satisfying the following formula are taken as the dangerous fruit pair:

[0028]

[0029] where (x i , y i , z i ) and (x j , y j , z j ) are the fruit coordinates in the collision region V D , and R m is the maximum working radius of the end effector.

[0030] Still further, the collision region V D is determined by the following method: extracting the fruit coordinates [x, y, z] LR and [x, y, z] MR1The coordinate (x1, y1, z1) with the largest y value, the fruit coordinate allocated to the right arm [x, y, z] RR And [x, y, z] MR2 The coordinate (x2, y2, z2) with the smallest y value, the collision area V of the left and right arms D Is set as the y coordinate The spatial region.

[0031] Further, the time T used by the mechanical arm to pick all the fruits in the transfer domain M Is calculated in the following manner:

[0032]

[0033] Wherein, S is the execution path length of the sequential planning of the transfer domain, n is the number of fruits in the transfer domain of the mechanical arm, K is the distance between the end of the mechanical arm and the x coordinate of the fruit, V m is the linear movement speed of the end of the mechanical arm, T d is the opening or closing time of the end effector.

[0034] Further, when calculating the picking execution time T L of the left arm and the picking execution time T R of the right arm, the shortest path of the left and right arms obtained by the genetic algorithm is taken as the execution path length of the sequential planning of the left and right arms, and the number of fruits is the number of fruits allocated to the left and right arms in the pre-allocation result.

[0035] The beneficial effects of the present application are as follows: the present application realizes rapid redundant fruit coordinate elimination through extraction of the hand-eye combined region of interest, divides the left / right arm exclusive domain and the double-arm transfer domain fruit based on the hand-eye combined region of interest, and realizes pre-allocation of the minimum difference value of the number of fruits allocated by the double arms; based on the pre-allocation result, the picking execution time of the left and right arms is calculated; in the process of calculating the picking execution time of the left and right arms, the shortest path of the left and right arms is obtained by the genetic algorithm, and is taken as the execution path length of the sequential planning of the left and right arms; in the process of obtaining the shortest path of the left and right arms, a strong constraint that the time difference from the starting point of the picking path sequential planning to the corresponding dangerous fruit is greater than T M / N3 is introduced; based on the size of the picking execution time of the left and right arms, the number of fruits of the left and right arms is re-allocated, and the shortest path of the fruits after re-allocation by the double-arm picking is finally obtained. The present application solves the problems of low utilization rate of the mechanical arm and low success rate of picking during double-arm robot picking operation, and the method is simple and reliable, and has strong practicability. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a schematic diagram of the picking robot according to the present application;

[0037] Figure 2A top view of the picking robot according to the present application;

[0038] Figure 3 A robot body coordinate system diagram according to the present application;

[0039] Figure 4 A single-arm exclusive domain and a two-arm handover domain diagram according to the present application;

[0040] Figure 5 A dangerous fruit distance judgment diagram according to the present application;

[0041] Figure 6 A picking robot two-arm task planning method flowchart according to the present application;

[0042] In the figure: 1. end effector A, 2. depth camera A, 3. end effector B, 4. depth camera B, 5. deep crown layer fruit tree, 6. right arm robot joint, 7. left arm robot joint. DETAILED DESCRIPTION

[0043] The present application will be further described below in conjunction with the accompanying drawings and specific embodiments, but the scope of protection of the present application is not limited thereto.

[0044] The picking robot two-arm task planning method according to the present application, which is characterized by a hand-eye combined region of interest extraction sub-method, a two-arm task pre-distribution sub-method and a time-optimal task planning sub-method, as shown in Figure 6 .

[0045] As shown in Figure 2 , to achieve full coverage picking of deep crown layer, two six-degree-of-freedom arms are installed on the robot close to the fruit tree side and at a distance L robot from the edge of the robot body, where L robot is the radius of the flange of the mechanical arm installation base; in the forward direction, the two six-degree-of-freedom arms are installed on the upper part of the robot body and are respectively at a distance R robot from the edge of the robot body, where R robot is the working radius of the six-degree-of-freedom arm. On this basis, as shown in Figure 3 , a robot body global coordinate system U is established with the depth direction as the X axis, the forward direction as the Y axis, the vertical upward direction as the Z axis, and the center point of the two mechanical arm bases as the origin O. In this embodiment, R robot = 860 mm, L robot = 70 mm.

[0046] In the eye-in-hand configuration, the depth camera B4 and the end effector B3 are installed on the left arm, and the depth camera A2 and the end effector A1 are installed on the right arm. The deep crown layer fruit tree 5 is identified and positioned by the depth camera B4 and the depth camera B2. For details of installation, see Figure 1 .

[0047] I. The flow of the hand-eye combined region of interest extraction sub-method

[0048] Convert all the center point coordinates obtained by the two cameras to the robot body global coordinate system U to generate the fruit center point camera identification coordinate set [x, y, z] camera In the robot body global coordinate system U, for the fruit with a fruit center distance A less than the distance D (where D is the average diameter of the fruit), it is determined to be the same fruit, thereby eliminating the repeatedly positioned fruits in the field of view images of the two cameras, and generating the fruit center point camera non-repeated identification coordinate set [x, y, z] nd .

[0049] For the fruit center point camera non-repeated identification coordinate set [x, y, z] nd , according to the mechanical arm working space V r and the left mechanical arm base coordinates (x left , y left , z left ), the right mechanical arm base coordinates (x right , y right , z right ), the fruits that do not satisfy formula (1) or formula (2) are regarded as redundant fruits outside the mechanical arm working space:

[0050] (x-x left ) 2 +(y-y left ) 2 +(z-z left ) 2 ≤R robot 2 (1)

[0051] (x-x right ) 2 +(y-y right ) 2 +(z-z right ) 2 ≤R robot 2 (2)

[0052] wherein (x, y, z) is the fruit coordinates, which come from the fruit center point camera non-repeated identification coordinate set [x, y, z] nd .

[0053] The fruit center point coordinates outside the mechanical arm workspace are removed, and a fruit center point hand-eye combined interest coordinate set [X, Y, Z] is generated all .

[0054] II. The flow of the dual-arm task pre-allocation sub-method

[0055] As shown in Figure 4 , by judging the fruit coordinates in the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all , the fruit coordinates satisfying formula (3) are taken as the fruit set [x, y, z] LR in the left arm exclusive domain:

[0056]

[0057] The fruit coordinates satisfying formula (4) are taken as the fruit set [x, y, z] RR in the right arm exclusive domain:

[0058]

[0059] The fruit coordinates satisfying formula (5) are taken as the fruit set [x, y, z] MR in the dual-arm handover domain:

[0060]

[0061] In this embodiment, the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all is: [(0.563, -1.234, -0.622), (0.463, -1.034, 0.622), (0.563, -0.934, 0.322), (0.563, -0.534, 0.522), (0.663, -0.234, 0.732), (0.666, -0.244, 0.732), (0.463, 0.255, 0.222), (0.363, 0.634, 0.423), (0.463, 0.734, 0.522), (0.763, 1.034, 0.622), (0.663, 1.334, 0.522), (0.563, 1.225, 0.786)].

[0062] The number of fruits in the left arm exclusive domain fruit coordinate set [x, y, z] LR is determined according to formula (3) and set as N1, the number of fruits in the right arm exclusive domain fruit coordinate set [x, y, z] RR is determined according to formula (4) and set as N2, and the number of fruits in the dual-arm handover domain fruit coordinate set [x, y, z] MRThe number of inner fruits is set as N3. In this embodiment, N1 is 6, N2 is 6, and N3 is 3.

[0063] The fruit coordinate set [x, y, z] of the left arm in the transition region is sorted according to the y value, and is divided into [x, y, z] MR The fruit coordinate set [x, y, z] of the right arm in the transition region is sorted according to the y value, and is divided into [x, y, z] MR1 The fruit coordinate set [x, y, z] of the right arm in the transition region is sorted according to the y value, and is divided into [x, y, z] MR2 The fruit coordinate set [x, y, z] of the left arm in the transition region is sorted according to the y value, and is divided into [x, y, z] MR1 The fruit coordinate set [x, y, z] of the right arm in the transition region is sorted according to the y value, and is divided into [x, y, z] MR2 The fruit coordinate set [x, y, z] of the right arm in the transition region is sorted according to the y value, and is divided into [x, y, z] all The fruit coordinate set [x, y, z] of the right arm in the transition region is sorted according to the y value, and is divided into [x, y, z]

[0064] If N1+N2+N3 is even, then:

[0065] If N1+N2+N3 is odd, then: Or

[0066] In this embodiment, the fruit coordinate set allocated to the left arm is [(0.563, -1.234, -0.622), (0.463, -1.034, 0.622), (0.563, -0.934, 0.322), (0.563, -0.534, 0.522), (0.663, -0.234, 0.732), (0.666, -0.244, 0.732)], and the fruit coordinate set allocated to the right arm is [(0.463, 0.255, 0.222), (0.363, 0.634, 0.423), (0.463, 0.734, 0.522), (0.763, 1.034, 0.622), (0.663, 1.334, 0.522), (0.563, 1.225, 0.786)].

[0067] III. The flow of the time-optimal task planning sub-method

[0068] For continuous picking of multiple fruits, the right arm robot joint 6 and the left arm robot joint 7 are kept outside during picking. First, the robot end is moved to the picking path sequence planning point (i.e., Kcm in front of the fruit x coordinate), the end effector is opened, it is horizontally advanced by Kcm, the end effector is closed to pick the fruit, it is horizontally retreated by Kcm, the end effector is opened to release the fruit, the end effector is closed again, and the end is kept moving linearly at a speed of V m, moves to the next execution fruit picking path sequence planning point, repeats execution until all coordinates are executed. The calculation method of the time used for picking all fruits is:

[0069]

[0070] wherein S is the length of the execution path of the sequence planning, n is the number of fruits, V m is the linear movement speed of the end of the mechanical arm, T d is the opening or closing time of the end effector; in this embodiment, V m is 0.2 m / s, T d is 1 s, and K is 10 cm.

[0071] For the fruit coordinate set [x, y, z] MR in the double-arm transfer domain, the sequence planning starting point is set as the coordinate point (x MR , y s , z s ) with the minimum x coordinate in the fruit coordinate set [x, y, z] s in the double-arm transfer domain, the shortest path is obtained by using a genetic algorithm, as the execution path length of the sequence planning of the transfer domain, and the time T M used for picking all fruits in the transfer domain is obtained in the manner of formula (6) (in the process of obtaining, n is the number of fruits in the mechanical arm transfer domain), and the average time used is T M / N3. In this embodiment, the average time used is 3.29 s.

[0072] According to the pre-allocation result, the left arm picking execution time T L and the right arm picking execution time T R are calculated. In the process of obtaining T L and T R , the sequence planning starting point of the left arm is set as the initial pose coordinate (x l , y l , z l ) of the end effector of the left mechanical arm, the sequence planning starting point of the right arm is set as the initial pose coordinate (x r , y r , z r ) of the end effector of the left mechanical arm, the shortest paths of the left and right arms are obtained by using a genetic algorithm, and are used as the execution path lengths of the sequence planning of the left and right arms, and the number of fruits is the pre-allocation result.

[0073] In the process of obtaining the shortest paths of the left and right arms, the time difference of the sequence planning starting point of the picking path reaching the corresponding dangerous fruit is greater than T MThe strong constraint of / N3 specifically involves: before each generation of crossover and mutation operations, determining the time difference between the left arm's initial pose and the right arm's initial pose and the right arm's time difference, and setting the time difference greater than or equal to T. M Remove the path corresponding to / N3, iterate to obtain the path that meets the time limit, and select the path with the smallest value among the paths that meet the time limit as the shortest path.

[0074] Dangerous fruits are defined as follows: for the collision area V D The relative spatial positions of the fruits assigned to different robotic arms are determined. All fruit coordinate pairs that satisfy equation (7) are considered dangerous fruit pairs (see [reference]). Figure 5 ):

[0075]

[0076] Among them, (x i ,y i ,z i ) and (x j ,y j ,z j ) represents the collision region V D Coordinates of the inner fruit; in this embodiment, the dangerous fruit pair is [(0.663, -0.234, 0.732), (0.666, -0.244, 0.732)].

[0077] Collision Zone V D This is determined by extracting the fruit coordinates [x, y, z] assigned to the left arm from the task allocation results. LR and [x,y,z] MR1 The coordinates of the fruit with the largest y-value (x1, y1, z1) and the coordinates of the fruit assigned to the right arm [x, y, z] RR and [x,y,z] MR2 The coordinates (x2, y2, z2) with the smallest y-value in the middle are used to determine the collision area V of the left and right arms. D Let the y-coordinate be... The spatial region, in which R m This represents the maximum working radius of the end effector.

[0078] Compare the left arm harvesting execution time T L And the right arm harvesting execution time T R The data is redistributed, allowing the robotic arm to quickly complete the harvesting task.

[0079] If the left arm retrieval was performed at time T L Greater than the right arm retrieval execution time T R Then the set of fruit coordinates [x,y,z] of the left arm exclusive domain in the pre-allocation result will be used.LR and the left arm part fruit coordinate set [x, y, z] in the handover region MR1 in the left arm part The fruit is assigned to the right arm, and the [x, y, z] is preferentially reassigned LR and [x, y, z] MR1 The fruit with the maximum y coordinate in the middle; if the left arm picking execution time T L is less than the right arm picking execution time T R , the right arm exclusive region fruit coordinate set [x, y, z] in the pre-assignment result RR and the right arm part fruit coordinate set [x, y, z] in the handover region MR2 in the right arm part The fruit is assigned to the left arm, and the [x, y, z] is preferentially reassigned RR and [x, y, z] MR2 The fruit with the minimum y coordinate in the middle, wherein ΔT is the difference between the left arm picking execution time and the right arm picking execution time.

[0080] According to the reassigned result, the collision region is regenerated, and the dangerous fruit pairs are detected, and the shortest path of the fruit after the dual-arm picking reassigned is obtained by using a genetic algorithm.

[0081] The embodiments are preferred embodiments of the present application, but the present application is not limited to the above embodiments, and any obvious improvement, replacement or modification made by those skilled in the art without departing from the essential content of the present application shall fall within the protection scope of the present application.

Claims

1.A method for task planning of a deep crown layer fruit distribution feature picking robot dual-arm, characterized in that: By judging the fruit coordinates in the set [x, y, z] of the fruit center point hand-eye combined interested coordinate set [X, Y, Z] all , the fruit set [x, y, z] in the left arm exclusive domain, the fruit set [x, y, z] in the right arm exclusive domain, and the fruit set [x, y, z] in the double arm transition domain are divided. LR , [x, y, z] RR , and [x, y, z] MR ; the fruit coordinates in the set [x, y, z] of the double arm transition domain fruit coordinate set [x, y, z] MR are sorted according to the size of the y value, and are divided into two regions [x, y, z] MR1 and [x, y, z] MR2 . With the parity of the number of fruits in the harvesting space, the set of coordinates of interest [X, Y, Z] of the center point of the hand-eye combination of fruits all The number of fruits in the inner part is pre-allocated according to the minimum number difference: If N1+N2+N3 is even, then: If N1+N2+N3 is odd, then: or where N1 is the number of fruit in the left arm exclusive domain [x, y, z] LR N2 is the number of fruit in the right arm exclusive domain [x, y, z] RR N3 is the number of fruit in the dual arm intersection domain [x, y, z] MR N4 is [x, y, z] MR1 N5 is [x, y, z] MR2 ; According to the pre-allocation result, the left arm picking execution time T L and the right arm picking execution time T R are calculated. In the calculation of T L and T R , the left arm sequential planning starting point is set as the initial pose coordinates (x l , y l , z l ) of the left arm end effector, the right arm sequential planning starting point is set as the initial pose coordinates (x r , y r , z r ) of the right arm end effector, and the shortest path of the left and right arms is calculated through a genetic algorithm; in the calculation of the shortest path of the left and right arms, a strong constraint that the time difference of the arrival of the picking path sequential planning starting point to the dangerous fruit is greater than T M / N3 is introduced; wherein T M is the time used for picking all fruits in the arm picking transfer domain. Compare the left arm harvesting execution time T L And the right arm harvesting execution time T R Size, redistribute fruits: if T L Greater than T R Then the set of fruit coordinates [x, y, z] of the left arm exclusive domain in the pre-allocation result will be used. LR And the set of fruit coordinates [x, y, z] MR1 Inside Each fruit is assigned to the right arm, with priority given to reallocating [x, y, z]. LR and [x, y, z] MR1 The fruit with the largest y-coordinate in T; if T L Less than T R Then the set of fruit coordinates [x, y, z] of the right arm exclusive domain in the pre-allocation result will be used. RR and [x, y, z] MR2 Inside Each fruit is assigned to the left arm, with priority given to reallocating [x, y, z]. RR and [x, y, z] MR2 The fruit with the smallest y-coordinate, where ΔT is the difference in harvesting time between the left and right arms; According to the reassignment result, the shortest path of the fruit after reassignment of the dual-arm picking is obtained by a genetic algorithm. 2.The picking robot dual-arm task planning method according to claim 1, characterized in that, The fruit center point hand-eye combined interest coordinate set [X, Y, Z] all By the following way: Convert all the center point coordinates obtained by the depth cameras installed on the two six-degree-of-freedom robot arms to the robot body global coordinate system U, generate the fruit center point camera recognition coordinate set [x, y, z] camera , remove the repeatedly positioned fruits within the field of view images of the two depth cameras, further generate the fruit center point camera non-repeated recognition coordinate set [x, y, z] nd , remove the redundant fruit center point coordinates outside the working space of the robot arm, and further generate the fruit center point hand-eye combined interest coordinate set [X, Y, Z] all . 3.The picking robot dual-arm task planning method according to claim 1, characterized in that, The set of coordinates of interest of the hand-eye combination of the center point of the fruit [X, Y, Z] all The fruit coordinates in the left arm exclusive domain [x, y, z] LR The fruit coordinates in the right arm exclusive domain [x, y, z] RR The fruit coordinates in the dual-arm transition domain [x, y, z] MR Specifically, The fruit coordinates satisfying the equation (1) are taken as the fruit set [x, y, z] in the left arm exclusive domain LR : The fruit coordinates satisfying Equation (2) are taken as the fruit set [x, y, z] within the right-arm exclusive domain RR : The fruit coordinates satisfying Equation (3) are taken as the fruit set [x, y, z] within the dual-arm handover domain MR : wherein (x left , y left , z left ) is the left robot base coordinate, (x right , y right , z right ) is the right robot base coordinate, and R robot is the robot working radius. 4.The picking robot dual-arm task planning method according to claim 1, characterized in that, The time difference between the arrival of the start of the introduced picking path sequence to the dangerous fruit and the dangerous fruit is greater than T M The strong constraint of / N3 is that, before each generation of cross and variation operation, the time difference between the time point when the left arm reaches the dangerous fruit on the left from the initial pose and the time point when the right arm reaches the dangerous fruit on the right from the initial pose is greater than a time difference and less than or equal to T M The path corresponding to / N3 is removed, and the path satisfying the time limit is obtained through iteration, and the path with the smallest value among the paths satisfying the time limit is selected as the shortest path. 5.The picking robot dual-arm task planning method according to claim 4, characterized in that, The dangerous fruit pair, consisting of the left and right dangerous fruits, is detected by the collision area V. D The relative spatial positions of the fruits assigned to different robotic arms are determined. All fruit coordinate pairs that satisfy the following formula are considered dangerous fruit pairs: where (x i , y i , z i ) and (x j , y j , z j ) are the fruit coordinates within the collision region V D , and R m is the maximum working radius of the end effector. 6.The picking robot dual-arm task planning method according to claim 5, characterized in that, The collision region V D The determination is made by extracting the coordinates [x, y, z] of the fruit to which the left arm is assigned in the pre-assignment result LR and the coordinates [x, y, z] of the fruit to which the right arm is assigned MR1 The coordinates (x1, y1, z1) with the largest y value among [x, y, z] RR and the coordinates (x2, y2, z2) with the smallest y value among [x, y, z] MR2 The left and right arm collision region V D is set as a spatial region in which the y coordinate is ​ 7.The picking robot dual-arm task planning method according to claim 1, characterized in that, the time T taken by the robot arm to pick all the fruits in the picking and handing over domain M is calculated by: where S is the execution path length of the handover domain sequential planning, n is the number of fruits in the handover domain of the robot arm, K is the distance between the end of the robot arm and the x coordinate of the fruit, V m is the linear movement speed of the end of the robot arm, T d is the opening or closing time of the end effector. 8.The picking robot dual-arm task planning method according to claim 7, characterized in that, Compute left arm picking execution time T L and right arm picking execution time T R When, the left and right arm shortest path solved by genetic algorithm are respectively taken as the execution path length of left and right arm sequential planning, and the fruit number is the number of left and right arm allocation in pre-allocation result.

Images