Ball-like target double-arm picking robot in gobi environment

Abstract

The invention provides a sphere-like target double-arm picking robot in a gobi environment, and relates to the field of intelligent picking, the sphere-like target double-arm picking robot comprises a collecting chassis (10), a picking device, a sorting device and a driving device (40), the collecting chassis (10) is of a cuboid box type structure, four cavities (13) are formed in the collecting chassis (10) through partition plates, and six mounting columns (11) are uniformly arranged on the bottom surface of the collecting chassis (10); driving devices (40) are respectively arranged at the bottom ends of the six mounting columns (11); two picking devices are arranged on the front side of the top end of the collecting chassis (10), and a sorting device is arranged between the two picking devices. The picking robot can effectively recognize mature spheroid fruits, the recognition accuracy is high, the mature spheroid fruits can be automatically picked, the fruit picking cost is reduced, the picking efficiency is improved, and the problems of mistaken picking, missing picking and the like in the existing picking process are effectively avoided.

Description

Technical Field

[0001] This invention relates to the field of intelligent harvesting technology, specifically to a dual-arm harvesting robot for spherical targets in a Gobi Desert environment. Background Technology

[0002] Currently, the harvesting of spherical fruits (such as apples) is still mostly done manually. However, with the increasing aging of the workforce, the advancement of agricultural modernization, and the large-scale outflow of rural labor, the cost of manual labor is gradually rising, and manual harvesting is inefficient and slow, resulting in poor uniformity of fruit ripeness and poor harvesting results.

[0003] In recent years, with the rapid development of robotics, vision, and sensor technologies, more and more harvesting robots have been gradually used in fruit picking, replacing traditional manual harvesting. This improves harvesting efficiency, reduces harvesting costs, and ensures the consistency of ripeness in the same batch of harvested fruit. Chinese patent document CN215345941 discloses a highly free-degree-of-freedom apple harvesting robot that can adjust the height of its gripper according to the height of the apple, thus facilitating the harvesting of apples at different locations. This solves the problem of low freedom of movement and inconvenience in adjustment of harvesting robots during operation. Simultaneously, it can adjust the position of the gripper according to the apple's location, allowing for harvesting operations at different positions, demonstrating good flexibility and improving the device's overall agility. However, this apple harvesting robot cannot effectively identify and avoid obstacles such as tree trunks, branches, and leaves, causing the end effector to be unable to harvest in a suitable posture, thus affecting harvesting efficiency and even potentially causing fruit damage during the harvesting process. Chinese patent document CN218218382U discloses a lightweight dual-arm apple-picking robot. It reduces the degrees of freedom of existing industrial 6-axis robotic arms to 3 degrees of freedom using a Cartesian coordinate robotic arm. The robot utilizes the linear motion of three linear slide modules (X, Z, Y) to locate and pick apples, improving the efficiency of apple picking. However, it suffers from poor environmental adaptability and an inability to effectively distinguish between ripe and unripe apples, leading to the mispicking of unripe apples and severely impacting the economic value of apple cultivation. Furthermore, the robot is prone to interference between the two robotic arms, further affecting picking efficiency.

[0004] In summary, while existing technologies include intelligent fruit-picking robots to replace manual fruit picking and improve picking efficiency and reduce costs, problems remain, such as difficulty in accurately identifying the fruit, susceptibility to external environmental factors during the picking process, and difficulty in effectively distinguishing ripe fruit. These issues lead to mispicking and missed picking, which not only slows down the picking efficiency but also easily causes irreversible damage to the fruit trees or fruit, thus affecting apple yield. Summary of the Invention

[0005] To address the problems existing in the prior art, the present invention aims to provide a dual-arm harvesting robot for spherical targets in the Gobi Desert environment. This harvesting robot can effectively identify mature spherical fruits with high accuracy and can autonomously harvest mature spherical fruits, thereby reducing fruit harvesting costs, improving harvesting efficiency, and effectively avoiding problems such as misharvesting and missed harvesting.

[0006] The objective of this invention is achieved through the following technical solution: A dual-arm harvesting robot for spherical targets in a Gobi Desert environment includes a collection chassis, harvesting devices, sorting devices, and a drive unit. The collection chassis has a rectangular box structure with four cavities inside separated by partitions and six mounting columns evenly arranged on its bottom surface. The drive unit is installed at the bottom of each of the six mounting columns. Two sets of harvesting devices are respectively installed on the front top of the collection chassis via a motor rotating platform, with a sorting device between the two sets of harvesting devices. The motor rotating platform is located on the top surface of the front cavity of the collection chassis, and the sorting devices correspond to the three cavities at the rear. A 3D LiDAR is installed at the front center of the motor rotating platform of the collection chassis, and a control module, an IMU, and a battery module are installed in the cavity corresponding to the motor rotating platform.

[0007] Based on further optimization of the above scheme, the harvesting device includes a lifting column, a base camera, a lifting screw, a lifting platform, a harvesting robotic arm, a harvesting claw, and a recognition camera. The lifting column is rotatably mounted on a motor rotating platform, and the inner cavity of the lifting column is hollow, with vertical sliding grooves communicating with the inner cavity on each of its four sides. The lifting screw is rotatably mounted inside the lifting column, and the base camera is mounted on its top end face. The lifting platform is slidably mounted inside the lifting column, and its middle part is threaded through by the lifting screw. The harvesting robotic arm is mounted on the outer wall of the front end of the lifting platform, and the harvesting claw is mounted on the end of the harvesting robotic arm away from the lifting platform. The recognition camera is mounted in the middle of the harvesting claw.

[0008] Based on further optimization of the above scheme, the picking robotic arm includes a mechanical forearm and a mechanical rear arm. One end of the mechanical forearm is connected to the lifting platform and the other end is connected to the mechanical rear arm. A picking claw is set at the end of the mechanical rear arm away from the mechanical forearm. Harmonic reducers are set at the connection points of the lifting platform, the mechanical forearm, and the mechanical rear arm.

[0009] Based on further optimization of the above scheme, the picking claw includes four claw mechanisms, a connecting turntable, a claw front shell, and a claw base. The four claw mechanisms include a claw, a first connecting rod, a second connecting rod, and a third connecting rod. The end of the claw is connected to the first connecting rod, and an anti-slip plate is provided on the inner side of the claw. A force sensor is installed between the anti-slip plate and the inner wall of the claw. The first connecting rod and the third connecting rod are connected through the second connecting rod, and the third connecting rod is connected to the connecting turntable. The connecting turntable is located in the claw front shell and controlled by a claw motor located in the claw front shell. A recognition camera is located in the end face of the claw front shell. The central axis of the connection between the third link and the second link and the connecting turntable is parallel to the central axis of the front shell of the gripper. The central axis of the connection between the first link and the second link and the gripper is perpendicular to the central axis of the front shell of the gripper. The end face of the front shell of the gripper has a movable groove corresponding to the gripper. The side of the front shell of the gripper away from the gripper is rotatably connected to the gripper base, and a rotary motor is fixedly installed on the gripper base. The side of the gripper base away from the front shell of the gripper is located at the end of the harvesting robot arm. Both the front shell of the gripper and the gripper base are cylindrical. A six-axis torque sensor and an end effector IMU are installed between the harvesting gripper and the harvesting robot arm.

[0010] Based on further optimization of the above scheme, the sorting device includes a conical collector, a sorting tube, and a sorting rotary drum mechanism. The conical collector is made of elastic fabric, with its upper end expanding outward and its lower end connected to the sorting tube. The sorting tube is an L-shaped flexible tube, and its end is connected to the sorting rotary drum mechanism. The sorting rotary drum mechanism includes four sorting rings and twelve sorting rods. The four sorting rings are arranged in three sections corresponding to the three cavities at the rear end of the collection chassis, and the four sorting rings are coaxially arranged. The first section has five sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The second section has four sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The third section has three sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The angle of the line connecting the centers of the sorting rings is 10° upward.

[0011] Based on further optimization of the above scheme, the drive device includes a main shaft, a main support frame, a drive shaft, a main drive wheel, a secondary wheel side plate, a secondary drive wheel, a shock-absorbing wheel, a hydraulic shock-absorbing rod, and a track. The main shaft is connected to the mounting column, and the main support frame is sleeved on the outer wall of the main shaft. The main support frame has an inverted T-shaped structure, and a drive shaft is set at its upper end. The main drive wheel is sleeved on the outer wall of the drive shaft. Secondary wheel side plates are set at both ends of the lower side of the main support frame. The secondary wheel side plates have a triangular structure, and secondary drive wheels and shock-absorbing wheels are set at their other two ends, respectively. Hydraulic shock-absorbing rods are set between the two shock-absorbing wheels and the main support frame, and the two shock-absorbing wheels have an inverted I-shaped structure. The track covers the outer wall of the main drive wheel and the two secondary drive wheels, and the track has a triangular structure.

[0012] A method for harvesting spherical fruits, including: Step S1: First, extract row directions and tree positions based on 3DLiDAR, generate candidate observation poses and expand them with small perturbations, and introduce occlusion sensitive field and picking window improvement index (TOWP-EI); then calculate the overlap, incident tolerance and predicted time-varying no-go region of the two picking robotic arms to obtain the optimal target pose and rolling replanning. Step S2: Within the allowable pose and time window obtained in step S1, first obtain spectral difference data, micro-torsional load compliance and fruit surface microstructure data, and construct a unified evidence function; then reduce the disturbance caused by environmental factors through uncertainty, thereby judging the degree of fruit ripeness. Step S3: Taking the optimal separation time as the target, and combining it with dynamic obstacle risk and end-effector alignment error, set the upper limit of the gripping force and the twisting speed of the picking claw, plan the smooth trajectory of the picking robotic arm, and at the same time maintain a safe distance between the two picking robotic arms to achieve fruit picking; during the picking process, monitor the force, posture and risk threshold in real time, and if an abnormality is found, immediately replan the picking process.

[0013] Based on further optimization of the above scheme, step S1, "extracting row direction and tree position based on 3DLiDAR, generating candidate observation poses and expanding with small perturbations, and introducing occlusion-sensitive field and picking window improvement index", specifically includes: First, point cloud data of the fruit trees between rows was acquired using 3D LiDAR, and then the set of center points of the tree trunks was obtained through clustering and curve fitting. c i} and row baseline; Then, a set of candidate observation poses is generated in front of the harvesting robot using the baseline of the line: ; In the formula: p k Indicates planar position; Indicates the robot's orientation; To improve the accessibility and robustness of the harvesting robot, at each candidate observation pose... Based on this, small perturbations are introduced to form an extended set. The final extended set, constrained by the inter-row channel width and the turning radius of the picking robot, is used as the input to the comprehensive cost function. And in each of the observed pose sets... x k A limited sampling of lateral offset and orientation perturbations is set around the perimeter to form a local pose cluster to obtain an optimized search space; Next, the occlusion-sensitive potential field is calculated: first, within the field of view of each candidate observation pose, the fruit tree point cloud data obtained by the 3D LiDAR sensor is sliced ​​and its near-canopy projection duty cycle is calculated. dk With structural uncertainty Then, in coordination with the robot, it moved towards Orientation deviation Obtain occlusion risk indicators: ; ; In the formula: Indicates the corresponding weight coefficient ( ); A occ (x k ) Represents the field of view x k Within the height range [ z min , z max ]Inside( z min , z max Based on the specific object, if the canopy layer is pre-calibrated, the projected area of ​​the bottom surface covered by the point cloud; A fov (x k ) Represents the field of view x k The corresponding total visible ground area; Var z (x k ) Represents the field of view x k The variance of the corresponding point cloud slice in the depth (i.e., z-axis) direction; This represents the baseline standard deviation of the calibration scenario; Represents the field of view x k The angular deviation between the actual orientation and the reference orientation; wrap Indicates the angle wrapping operation; The range of observed poses to be analyzed is discretized to obtain a series of candidate poses. x k}, ensuring coverage of the spatial range of interest; for each candidate pose, obtain its occlusion risk index. Finally, the occlusion risk index of discrete candidate poses is extended to a continuous spatial field (e.g., by using spatial interpolation or field modeling methods) to obtain the occlusion-sensitive latent field (i.e., the occlusion-sensitive latent field is a continuous spatial mapping of the occlusion risk index). Next, by constructing the Time-of-Harvest Improvement Index (TOWP-EI), the time of maximum separation due to subsequent shading is predicted: ; In the formula: clip Represents the amplitude limiting function; This represents the time required to reach maximum separation (i.e., stable picking state) after the self-occlusion excitation is initiated under the reference pose condition; This indicates the prediction time window (typically 1.5 seconds). D(x k ) This represents the distance parameter related to the observed pose; w k This represents the estimated value of the local crosswind. This indicates the angle between the camera and the fruit stand; These represent the corresponding weight coefficients; Quantifying the observed pose using the Time-of-Harvest Improvement Index (TOWP-EI). x k The larger the index, the faster a stable harvestable window can be formed on the surface.

[0014] Based on further optimization of the above scheme, the step of […] in each candidate observation pose... Based on this, a small perturbation is introduced to form an extended set. The final extended set, after being constrained by the width of the inter-row passage and the turning radius of the picking robot, is used as the input to the comprehensive cost function, specifically: Small perturbation spread of candidate observation pose: For each observation pose In its base coordinate system B k (thinking it was facing) Introducing disturbances within: ; ; In the formula: These represent disturbances in the longitudinal, lateral, and directional directions, respectively. All perturbed poses are incorporated into the extended set: ; Where: perturbation domain Symmetrical, zero-mean small-range sampling is used to enhance accessibility and robustness; Feasibility constraints on aisle width and minimum turning radius: The effective width of the aisle between rows is W y The robot's maximum lateral width is Wr The safety margin is m s Then the lateral perturbation satisfies: ; The robot's minimum turning radius is R min For any from x k arrive The short-range connection trajectory (this trajectory is connected to the arc length of) L h (approximate to a circular arc), its curvature satisfy: ; Under the small perturbation approximation, the terminal orientation changes. With arc length L h The ratio approximates the upper bound of curvature: ; In the formula: This represents the maximum angle that the robot's orientation is allowed to change within each local path segment; Based on the above constraints, Feasible poses that do not meet the requirements for lateral movement and turning radius are filtered to obtain the final extended set: ; The observed poses in this set serve as the subsequent comprehensive cost function. J(x k ) The input candidate points are used for path planning and pose optimization.

[0015] Based on further optimization of the above scheme, each of the observed poses in the set... x k A finite sampling method is used to set up lateral offsets and orientation perturbations around the model to form a local pose cluster, thereby obtaining the optimization search space. Specifically, for each of the observed poses in the set... Limited sampling is performed around the area to form a local pose cluster, where the lateral offset In scope The inner step size is fixed. Equal-interval sampling, towards the disturbance In scope According to the angle step size sampling; For each candidate pose x k Generate a sample set: ; Each of them The local pose cluster that constitutes the observed pose is used to form the optimization search space; the sampling range of lateral offset and orientation perturbation is jointly determined by the channel width constraint and the robot's minimum turning radius constraint, thus forming a local pose cluster to obtain the optimization search space.

[0016] Based on further optimization of the above scheme, step S1, "calculating the overlap, incident tolerance, and predicted time-varying no-go region of the two picking robotic arms to obtain the optimal target pose and rolling replanning," specifically involves: First, set the left and right picking robotic arms in the observation position. x k The reachable areas are as follows: R L (x k ) , R R (x k ) Obtain the overlap of the workspace of the harvesting robotic arm: ; In the formula: area Indicates the area of ​​the region; Incident angle tolerance y k Used to represent the observed pose of the occlusion decoupling terminal. x k The effective angle of incidence for the harvesting target is determined. First, the angle of incidence of the harvesting claw tip relative to the harvested fruit is obtained based on the current pose of the harvesting claw. Then, the angle of incidence tolerance is obtained through the feasible angle range for unobstructed harvesting. Finally, an incident penalty factor is introduced. Pen inc : ; In the formula: y min The minimum tolerance threshold representing the angle of incidence; Next, the time-varying no-entry region is predicted: based on the shading-sensitive potential field, the branching model, and wind disturbance estimation, in the time domain... Internal prediction may cause time-varying obstacle sets for occlusion B(t) Introduce a soft constraint penalty term for collision risk: ; In the formula: This represents the minimum distance between the trajectory and the branch projection; Indicates the scale parameter; Indicates a time delay; This represents the dynamic weighting coefficients; using the above model, the robot can dynamically update them within each scrolling time domain. It automatically adjusts the navigation trajectory based on the prediction results to ensure that the boom avoids the area blocked by branches and leaves that sway in the wind during the operation. The prediction of time-varying forbidden regions is used to provide a safety margin for subsequent gating trajectories with respect to time windows; Finally, the comprehensive cost function is obtained based on the candidate observation pose set. J(x k ) : ; In the formula: w 1. w 2. w 3. w 4. w 5. w 6 represents the corresponding weight coefficients; Next navigation target location x * From the comprehensive cost function J(x k ) Decision (even if it is necessary) J(x k ) (Minimum pose), and simultaneously, a rolling temporal optimization strategy is adopted, that is, the time-varying obstacle set is updated immediately after receiving a frame of data from the 3D LiDAR sensor and the IMU. B(t) With the maximum separation time And regain the comprehensive cost function. J(x k ) Finally, under the conditions of safety and the reachability of the picking robot, the next navigation path is obtained until the picking robot enters the predetermined observation pose point.

[0017] Based on further optimization of the above scheme, the specific steps in step S2, namely "within the allowable pose and time window obtained in step S1, first obtain spectral difference data, micro-torsional load compliance, and fruit surface microstructure data, and then uniformly construct an evidence function", are as follows: First, the observation pose given in step S1 x k With time window The spectral and surface imaging processes are completed internally, and the end effector of the right-side harvesting robotic arm does not exceed the preset safety amplitude. and normal force F max Under the condition of micro-amplitude torsional frequency sweep to obtain mechanical response; if time-varying obstacle set B(t) If the risk exceeds the set threshold or the time window shrinks, the current action will be terminated and the user will wait for the next time window. Subsequently, the relative reflectance was obtained under annular, narrowband light source conditions. R 680 , R 730 , R 850 Obtain the chlorophyll degradation difference index S sp : ; Using reflectivity R 850 Brightness normalization and shadow compensation were performed, and the chlorophyll degradation difference index was calculated. S sp As the fruit ripens, the red blush on the surface tends to increase monotonically, which is used to suppress the interference of the red blush on the green base. Then, mechanical compliance is obtained, and the end effector of the harvesting robot arm is excited by angular displacement. By applying a small-amplitude torsional measurement torque and the signal value from the IMU, the frequency domain angular compliance can be obtained. : ; In the formula: Represents angular frequency The excitation amplitude; Represents the set of angular frequencies; The frequency domain amplitude represents the angular displacement response; This represents the frequency domain amplitude of the torque excitation. Estimating the principal resonance frequency using a second-order equivalent model f r With damping ratio Thus, indicators of mechanical maturity are obtained. S mech : ; In the formula: These represent the weighting coefficients; This indicates the upper limit of the calibrated damping ratio; Mature fruits exhibit higher low-frequency compliance and lower resonant frequency / damping difference (i.e. S mech Increase); Next, the microstructure optical parameters of the surface (such as cuticle density, waxiness, etc.) are processed: the harvesting robot uses a structured light sensor to image and obtain the width of the fruit's specular reflection lobes and the roughness of the fruit's microtexture, and then uses the full width at half maximum (FWHM) to analyze the data. w spec Constructing optical indices of fruit surface microstructure using speckle contrast K S surf : ; The maturation of the fruit's cuticle and wax layer leads to sharpening of the speckled lobes and a decrease in speckled contrast (i.e., optical indicators of the fruit's surface microstructure). S surf (Increase).

[0018] Based on further optimization of the above scheme, step S2, "further reducing the disturbance caused by environmental factors through uncertainty, thereby judging the degree of fruit ripeness," specifically means: The fruit maturity evidence function is obtained through nonlinear fusion. S mat : ; In the formula: These represent the corresponding weight coefficients; These represent the indices obtained after normalization of the chlorophyll degradation difference index, the mechanical maturity index, and the optical index of the fruit surface microstructure, respectively; U represents the uncertainty measure. ; In the formula: u 1 represents the variance of repeated sampling, measured within a time window. The sample was repeated three times, and the ratio of standard deviation to mean was used to obtain the result. u 2 indicates dual-field consistency, determined by the maximum separation time difference between BaseCam and EndCam (BaseCam is the base camera, EndCam is the recognition camera). If the maximum separation time difference is no greater than 0.12s, the observation results of the two cameras are considered to be consistent; otherwise, the difference is normalized to one. u 3 indicates stability within the time window, i.e., the fluctuation range of the evidence function (maximum value - minimum value) / mean within the time window. ); Final output maturity score With confidence level : ; when and If the fruit is ripe, it is considered ripe; otherwise, it is considered unripe or uncertain.

[0019] Based on further optimization of the above scheme, step S3 specifically includes: First, generate the time-window gating trajectory: preset the initial pose. q(t 0 ) Picking posture is q pick The trajectory of the right arm of the harvesting robot was parameterized using joint space splines: ; ; In the formula: a j The weighting coefficients represent the basis functions of the trajectory. Indicates the first j One time base function; n eef Indicates the direction of the screwing shaft at the end of the picking hand. n ped Indicates the normal direction of the fruit stalk; Indicates the angle between two directions; This indicates the alignment tolerance at the end of the picking hand; The comprehensive cost function is then obtained: ; In the formula: w t , w j , w r , w n These represent the corresponding weight coefficients; This indicates the actual maximum separation time observed. X(t) Indicates the position of the picking claw tip; Subsequently, mechanical parameter feedforward and end-effector adaptive force control are performed: the set of mechanical parameters output in step S2 is used, including frequency domain angular compliance. Main resonant frequency f r With damping ratio This enables feedforward tuning of the gripper and selection control: The picking claw's gripping force is adaptive, and the maximum upper limit of the normal force is: ; In the formula: F base , F safe These represent the clamping reference force and the clamping safety force, respectively. This represents a normalized mapping based on the training samples; ; The picking claw uses force-position control, and during the closing process... Contact compliance serves as feedback to avoid over-force. Harvesting claw twisting frequency matching: The twisting angular velocity and excitation frequency of the harvesting claw are determined by... Adaptive settings are used to ensure that the natural frequency of the fruit stem connection system is maintained, thereby further reducing the torque required for separation and the risk of the peel being sheared. Throughout the harvesting process, it is essential to ensure coordinated operation of both arms and a safe hand-switching domain. The left arm of the harvesting robot is designated for shielding, decoupling, and auxiliary support, while the right arm performs the harvesting. To prevent interference between the two arms, constraints on the safe distance and reachability of both arms are defined. ; In scenarios requiring a hand-changing position, introduce the hand-changing posture. q swap With the moment of change of hands t swap And apply derivative continuity: ; In the formula: , These represent the left and right limits at the moment of hand change, respectively; Regarding triggering, online monitoring, and exception handling, the triggering conditions are as follows: ; In the formula: t int Indicates the risk score threshold; And it satisfies the safety domain constraints of the two arms of the harvesting robotic arm; If the following abnormal states occur in the end-effector force-displacement or torque-angular displacement curves during the entire process, the harvesting robot arm will immediately release its grip and return to the pre-position, waiting for a replanning trigger or postponing to the next time window: (1) The slope of the torque-angular displacement curve exceeds the threshold: ; (2) Terminal posture deviation or ;in, Indicates the end-effector pose deviation threshold; (3) Force control exceeding limits: ; (4) Risk Incentive Control: .

[0020] The following are the technical effects of the present invention: This invention utilizes a dual-arm harvesting robot comprised of a collection chassis, harvesting device, sorting device, and drive device. The collection chassis features a four-cavity segmented structure, working in conjunction with the sorting device to achieve an integrated layout of "control-power-sorting-collection." Furthermore, it effectively divides the harvesting process according to fruit size, avoiding the need for re-sorting after collection and improving harvesting efficiency. Simultaneously, the "T-shaped main support frame + triangular track" structure, combined with "hydraulic shock-absorbing rods + tilting I-beam shock-absorbing wheels," effectively adapts to complex terrains such as gravel roads and slopes in the Gobi Desert, reducing interference from bumps on sensors. The tracks cover the main and auxiliary drive wheels, increasing the ground contact area and friction to prevent slippage. The harvesting device, consisting of a lifting column, base camera, lifting screw, lifting platform, harvesting robotic arm, harvesting claw, and recognition camera, can adjust the harvesting end according to the fruit's posture and position, ensuring effective harvesting and preventing missed or incorrect harvesting.

[0021] This invention establishes a judgment basis through a triple index of "differential analysis + mechanical compliance + surface microstructure," accurately identifying and judging fruit maturity from chemical, physical, and morphological dimensions, significantly reducing the probability of errors and missed harvesting. Simultaneously, by fusing data and introducing uncertainty, it effectively offsets interference from light fluctuations and wind disturbances in the Gobi Desert environment, ensuring improved accuracy in maturity judgment, avoiding the wrong harvesting of immature fruit, and safeguarding the economic value of planting. Furthermore, this invention achieves efficient collaborative work between two harvesting robotic arms through spatial-temporal dual-dimensional planning, avoiding problems such as mutual interference and weak obstacle avoidance capabilities between harvesting robotic arms. Through mechanical control and structural optimization, utilizing adaptive force control, posture matching, and the flexible structure of the harvesting claw, this invention ensures effective fruit grasping and harvesting while avoiding fruit damage caused by uncontrolled gripping force or improper end-effector posture, ensuring a high yield of high-quality fruit after harvesting. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall structure of the harvesting robot in an embodiment of the present invention.

[0023] Figure 2 This is a schematic diagram of the collection chassis of the harvesting robot in an embodiment of the present invention.

[0024] Figure 3 This is a schematic diagram of the sorting device of the picking robot in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of the harvesting device (excluding the harvesting claw) of the harvesting robot in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the harvesting claw of the harvesting robot in an embodiment of the present invention; wherein, Figure 5 (a) is an overall schematic diagram. Figure 5 (b) is a schematic diagram of the internal structure of the foreshell without the claws. Figure 5 (c) is a sectional view.

[0027] Figure 6 This is a schematic diagram of the drive device of the harvesting robot in an embodiment of the present invention; wherein, Figure 6 (a) is a schematic diagram of the external structure. Figure 6 (b) is a schematic diagram of the internal structure.

[0028] Among them, 10. Collection chassis; 11. Mounting column; 12. 3D LiDAR; 13. Cavity; 21. Lifting column; 22. Base camera; 23. Lifting screw; 24. Lifting platform; 251. Mechanical forearm; 252. Mechanical rear arm; 2611. Gripper; 2612. First link; 2613. Second link; 2614. Third link; 262. Linkage turntable; 263. Gripper front shell; 264. Gripper base; 27. Identification camera; 31. Conical collector; 32. Sorting tube; 331. Sorting ring; 332. Sorting rod; 40. Drive unit; 41. Main shaft; 42. Main support frame; 43. Drive shaft; 44. Main drive wheel; 45. Secondary wheel side plate; 46. Secondary drive wheel; 47. Shock-absorbing wheel; 48. Hydraulic shock-absorbing rod; 49. Track. Detailed Implementation

[0029] The technical solutions in the embodiments of the present invention will be clearly and completely described below. In the following description, specific details such as specific system structures and technologies are presented for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention.

[0030] Example 1: A dual-arm harvesting robot for spherical targets in a Gobi Desert environment includes a collection chassis 10, a harvesting device, a sorting device, and a drive device 40 (e.g., Figure 1 As shown), the collecting chassis 10 has a rectangular box structure (as shown). Figure 1 or Figure 2 As shown), its interior is provided with four cavities 13 by partition plates, and six mounting columns 11 are evenly arranged on its bottom surface. A drive unit 40 is respectively installed at the bottom end of each of the six mounting columns 11. The drive unit 40 includes a main shaft 41, a main support frame 42, a drive shaft 43, a main drive wheel 44, a secondary wheel side plate 45, a secondary drive wheel 46, a shock-absorbing wheel 47, a hydraulic shock-absorbing rod 48, and tracks 49 (as shown). Figure 6 As shown), the main shaft 41 is connected to the mounting column 11, and the outer wall of the main shaft 41 is sleeved with the main support frame 42. The main support frame 42 is an inverted T-shaped structure (as shown). Figure 6(a) As shown, a drive shaft 43 is provided at its upper end, and a main drive wheel 44 is sleeved on the outer wall of the drive shaft 43. Secondary wheel side plates 45 are respectively provided at both ends of the lower side of the main support frame 42. The secondary wheel side plates 45 are triangular structures (e.g., ...). Figure 6 (a) As shown, secondary drive wheels 46 and shock absorber wheels 47 are respectively provided at its other two ends. Hydraulic shock absorber rods 48 are provided between the two shock absorber wheels 47 and the main support frame 42 in pairs (that is, three hydraulic shock absorber rods 48 are provided on one drive device 40, such as...). Figure 6 (b) As shown, the two shock-absorbing wheels 47 have an inverted I-shaped structure (as shown in the image). Figure 6 (b) shows); the track 49 covers the outer wall of the main drive wheel 44 and the two auxiliary drive wheels 46, and the track 49 has a triangular structure (as shown in the image). Figure 6 (As shown).

[0031] Collect the front side of the top of chassis 10 (i.e. Figure 2 The left top surface shown is equipped with two sets of picking devices via a motor-driven rotating platform, with a sorting device between the two sets of picking devices. Each picking device includes a lifting column 21, a base camera 22, a lifting screw 23, a lifting platform 24, a picking robotic arm, picking claws, and a recognition camera 27. The lifting column 21 is rotatably mounted on the motor-driven rotating platform, and its inner cavity is hollow, with vertical grooves (such as...) opening on its four sides that communicate with the inner cavity. Figure 4 As shown), a lifting screw 23 is rotatably installed inside the lifting column 21, and a base camera 22 is installed on its top end face (as shown). Figure 4 As shown), the lifting platform 24 is slidably disposed within the cavity of the lifting column 21, with its middle section threaded through by the lifting screw 23; a harvesting robotic arm is disposed on the outer wall of the front end of the lifting platform 24, and a harvesting claw is disposed at the end of the harvesting robotic arm away from the lifting platform, with a recognition camera 27 disposed in the middle of the harvesting claw. The harvesting robotic arm includes a robotic forearm 251 and a robotic rear arm 252 (as shown). Figure 4 As shown), one end of the mechanical forearm 251 is connected to the lifting platform 24, and the other end is connected to the mechanical rear arm 252. A picking claw is installed at the end of the mechanical rear arm 252 away from the mechanical forearm 251. Harmonious reducers are installed at the connection points of the lifting platform 24, the mechanical forearm 251, and the mechanical rear arm 252. The picking claw includes four gripper mechanisms, a connecting turntable 262, a gripper front shell 263, and a gripper base 264. The four gripper mechanisms include a gripper 2611, a first connecting rod 2612, a second connecting rod 2613, and a third connecting rod 2614 (as shown). Figure 5 As shown), the end of the gripper 2611 is connected to the first connecting rod 2612, and an anti-slip plate is provided on the inner side of the gripper 2611. A force sensor is installed between the anti-slip plate and the inner wall of the gripper 2611 (both the anti-slip plate and the force sensor can be existing products). The first connecting rod 2612 and the third connecting rod 2614 are connected through the second connecting rod 2613, and the third connecting rod 2614 is connected to the connecting rod turntable 262 (as shown). Figure 5(b) As shown, the connecting turntable 262 is disposed in the front housing of the gripper 2611 and controlled by the gripper motor disposed in the front housing 263 of the gripper, and the recognition camera 27 is disposed in the middle of the end face of the front housing 263 of the gripper (as shown). Figure 5 (a) As shown); the central axis of the connection between the third link 2614 and the second link 2613 and the connecting turntable 262 is parallel to the central axis of the front shell 263 of the gripper, and the central axis of the connection between the first link 2612 and the second link 2613 and the gripper 2611 is perpendicular to the central axis of the front shell 263 of the gripper (as shown in (a)). Figure 5 (c) As shown, the end face of the front shell 263 of the gripper has an active groove corresponding to the gripper 2611; the side of the front shell 263 away from the gripper 2611 is rotatably connected to the gripper base 264 and a rotary motor is fixedly installed on the gripper base 264; the side of the gripper base 264 away from the front shell 263 is located at the end of the harvesting robot arm (i.e., the mechanical forearm 251); both the front shell 263 and the gripper base 264 are cylindrical; a six-axis torque sensor and an end effector IMU are installed between the harvesting gripper and the harvesting robot arm (both the torque sensor and the IMU are existing products).

[0032] The sorting device includes a conical collector 31, a sorting tube 32, and a sorting rotary drum mechanism. The conical collector 31 is made of elastic fabric, with its upper end expanding outwards and its lower end connected to the sorting tube 32 (e.g., Figure 3 As shown), the sorting tube 32 is an L-shaped flexible tube, the end of which is connected to the sorting rotary drum mechanism; the sorting rotary drum mechanism includes four sorting rings 331 and twelve sorting rods 332. The four sorting rings 331 are arranged in three sections corresponding to the three cavities 13 at the rear end of the collection chassis 10, and the four sorting rings 331 are coaxially arranged (in conjunction with...). Figure 2 and Figure 3 As shown), the first section has five sorting rods 332 evenly distributed around the outer ring of the sorting ring 331. The second section has four sorting rods 332 evenly distributed around the outer ring of the sorting ring 331. The third section has three sorting rods 332 evenly distributed around the outer ring of the sorting ring 331. The angle of the line connecting the centers of the sorting rings 331 is 10° upwards (e.g., ...). Figure 3 (As shown).

[0033] The motor rotation platform is set on the top surface of the front cavity 13 of the collection chassis 10, and the sorting device corresponds to the three cavities 13 at the rear. A 3D LiDAR 12 is set at the front of the middle part of the motor rotation platform of the collection chassis 10, and a control module, IMU and battery module (all of which can be existing integrated modules) are set in the cavity 13 corresponding to the motor rotation platform.

[0034] Example 2: As another preferred embodiment of the present invention, in order to avoid collision and damage to the sidewalls of the cavity 13 of the collecting tray 10 during the fruit collection process, based on the embodiment 1, a sponge layer is evenly laid on the sidewalls and bottom surface of the last three cavities 13 of the collecting tray 10 to form a buffer.

[0035] Example 3: A method for harvesting spherical fruits, employing a harvesting robot as described in Example 1 or Example 2, includes: Step S1: First, extract row orientation and tree position based on 3D LiDAR, generate candidate observation poses and expand them with small perturbations, and introduce the occlusion-sensitive field and the picking window improvement index (TOWP-EI); specifically: First, point cloud data of the fruit trees between rows was acquired using 3D LiDAR, and then the set of center points of the tree trunks was obtained through clustering and curve fitting. c i} and row baseline; Then, a set of candidate observation poses is generated in front of the harvesting robot using the baseline of the line: ; In the formula: p k Indicates planar position; Indicates the robot's orientation; To improve the accessibility and robustness of the harvesting robot, at each candidate observation pose... Based on this, a small perturbation is introduced to form an extended set. The final extended set, after being constrained by the width of the inter-row passage and the turning radius of the picking robot, is used as the input to the comprehensive cost function; specifically: Small perturbation spread of candidate observation pose: For each observation pose In its base coordinate system B k (thinking it was facing) Introducing disturbances within: ; ; In the formula: These represent disturbances in the longitudinal, lateral, and directional directions, respectively. All perturbed poses are incorporated into the extended set: ; Where: perturbation domain Symmetrical, zero-mean small-range sampling is used to enhance accessibility and robustness; Feasibility constraints on aisle width and minimum turning radius: The effective width of the aisle between rows is W y The robot's maximum lateral width is W r The safety margin is m s Then the lateral perturbation satisfies: ; The robot's minimum turning radius is R min For any from x k arrive The short-range connection trajectory (this trajectory is connected to the arc length of) L h (approximate to a circular arc), its curvature satisfy: ; Under the small perturbation approximation, the terminal orientation changes. With arc length L h The ratio approximates the upper bound of curvature: ; In the formula: This indicates the maximum angle that the robot's orientation is allowed to change within each local path segment; Based on the above constraints, Feasible poses that do not meet the requirements for lateral movement and turning radius are filtered to obtain the final extended set: ; The observed poses in this set serve as the subsequent comprehensive cost function. J(x k ) The input candidate points are used for path planning and pose optimization.

[0036] And in each of the observed pose sets x k A finite sampling method is used to set lateral offsets and orientation perturbations around the model to form a local pose cluster, thereby obtaining the optimization search space; specifically: in each of the observed pose sets... Limited sampling is performed around the area to form a local pose cluster, where the lateral offset In scope The inner step size is fixed. Equal-interval sampling, towards the disturbance In scope According to the angle step size sampling; For each candidate pose x k Generate a sample set: ; Each of them The local pose cluster that constitutes the observed pose is used to form the optimization search space; the sampling range of lateral offset and orientation perturbation is jointly determined by the channel width constraint and the robot's minimum turning radius constraint, thus forming a local pose cluster to obtain the optimization search space.

[0037] Next, the occlusion-sensitive potential field is calculated: first, within the field of view of each candidate observation pose, the fruit tree point cloud data obtained by the 3D LiDAR sensor is sliced ​​and its near-canopy projection duty cycle is calculated. d k With structural uncertainty Then, in coordination with the robot, it moved towards Orientation deviation Obtain occlusion risk indicators: ; ; In the formula: Indicates the corresponding weight coefficient ( In this embodiment, the values ​​are 0.5, 0.3, and 0.2 respectively. A occ (x k ) Represents the field of view x k Within the height range [ z min , z max ]Inside( z min , z max Based on the specific object, if the canopy layer is pre-calibrated, the projected area of ​​the bottom surface covered by the point cloud; A fov (x k ) Represents the field of view x k The corresponding total visible ground area; Var z (x k ) Represents the field of view x k The variance of the corresponding point cloud slice in the depth (i.e., z-axis) direction; This represents the baseline standard deviation of the calibration scenario; Represents the field of view x kThe angular deviation between the actual orientation and the reference orientation; wrap Indicates the angle wrapping operation; The range of observed poses to be analyzed is discretized to obtain a series of candidate poses. x k}, ensuring coverage of the spatial range of interest; for each candidate pose, obtain its occlusion risk index. Finally, the occlusion risk index of discrete candidate poses is extended to a continuous spatial field (e.g., by using spatial interpolation or field modeling methods) to obtain the occlusion-sensitive latent field (i.e., the occlusion-sensitive latent field is a continuous spatial mapping of the occlusion risk index). Next, by constructing the Time-of-Harvest Improvement Index (TOWP-EI), the time of maximum separation due to subsequent shading is predicted: ; In the formula: clip Represents the amplitude limiting function; This represents the time required to reach maximum separation (i.e., stable picking state) after the self-occlusion excitation is initiated under the reference pose condition (1.8s in this embodiment). This indicates the prediction time window (typically 1.5 seconds). D(x k ) This represents the distance parameter related to the observed pose; w k This represents the estimated value of the local crosswind. This indicates the angle between the camera and the fruit stand; These represent the corresponding weighting coefficients (obtained by fitting experimental data); Quantifying the observed pose using the Time-of-Harvest Improvement Index (TOWP-EI). x k The larger the index, the faster a stable harvestable window can be formed on the surface.

[0038] Then, the overlap, incident tolerance, and predicted time-varying no-go region of the two picking robotic arms are calculated to obtain the optimal target pose and rolling replanning; specifically: First, set the left and right picking robotic arms in the observation position. x k The reachable areas are as follows: R L (x k ) , R R (x k ) Obtain the overlap of the workspace of the harvesting robotic arm: ; In the formula: area Indicates the area of ​​the region; Incident angle tolerance y k Used to represent the observed pose of the occlusion decoupling terminal. x k The effective angle of incidence for the harvesting target is determined. First, the angle of incidence of the harvesting claw tip relative to the harvested fruit is obtained based on the current pose of the harvesting claw. Then, the angle of incidence tolerance is obtained through the feasible angle range for unobstructed harvesting. Finally, an incident penalty factor is introduced. Pen inc : ; In the formula: y min This represents the minimum tolerance threshold for the angle of incidence (in this embodiment, y min (for 25°) Next, the time-varying forbidden region is predicted: based on the shading-sensitive potential field, the branch pendulum model, and wind disturbance estimation, where the branch pendulum model is based on simplified flexible rod dynamics, modeling a single branch as having an equivalent length. L b Linear density Equivalent stiffness With damping The forced vibration beam, its swing angle satisfy: ; in: The equivalent moment of inertia of the branch; This represents the equivalent wind force acting on the branches; Branches in their initial state The angular displacement solution after wind excitation is: ; In the formula: , representing the natural angular frequency of the branch; , indicating the damping ratio; Indicates the peak angle of wind load; Wind disturbance estimation was obtained jointly from IMU and 3D LiDAR point cloud drift data mounted on the motor rotating platform; the IMU in time t i The transverse angular velocity is The spatial drift vector of LiDAR between adjacent frames is Then, the local average wind speed and instantaneous wind direction are obtained: ; according to and The time series data was processed using moving average and first-order filtering to obtain a continuous wind speed function. Its excitation force is approximated by aerodynamics: ; In the formula: Indicates air density, Indicates the drag coefficient. This indicates the area of ​​the branches and leaves exposed to the wind.

[0039] In the time domain Internal prediction may cause time-varying obstacle sets for occlusion B(t) (In this embodiment, (1.5s), introducing soft constraint penalties for collision risk. Pen dyn : ; In the formula: This represents the minimum distance between the trajectory and the branch projection; Indicates the scale parameter; Indicates a time delay; This represents the dynamic weighting coefficient (0.8 in this embodiment); using the above model, the robot can dynamically update the weighting coefficient within each rolling time domain. It automatically adjusts the navigation trajectory based on the prediction results to ensure that the boom avoids the area blocked by branches and leaves that sway in the wind during the operation. The prediction of time-varying forbidden regions is used to provide a safety margin for subsequent gating trajectories with respect to time windows; Finally, the comprehensive cost function is obtained based on the candidate observation pose set. J(x k ) : ; In the formula: w 1. w 2. w 3. w 4. w 5. w 6 represents the corresponding weight coefficients (in this embodiment, they are 1.0, 2.0, 3.0, 1.5, 1.0, and 2.5 respectively). Next navigation target location x * From the comprehensive cost function J(x k ) Decision (even if it is necessary) J(x k )(Minimum pose), and simultaneously, a rolling temporal optimization strategy is adopted, that is, the time-varying obstacle set is updated immediately after receiving a frame of data from the 3D LiDAR sensor and the IMU. B(t) With the maximum separation time And regain the comprehensive cost function. J(x k ) Finally, under the conditions of safety and the reachability of the picking robot, the next navigation path is obtained until the picking robot enters the predetermined observation pose point.

[0040] Step S2: Within the allowable pose and time window obtained in step S1, first obtain spectral difference data, micro-torsional load compliance data, and fruit surface microstructure data, and then construct a unified evidence function; specifically: First, the observation pose given in step S1 x k With time window The spectral and surface imaging processes are completed internally, and the end effector of the right-side harvesting robotic arm does not exceed the preset safety amplitude. and normal force F max (In this embodiment, Not greater than 1.5 F max Under the condition of not exceeding 6), perform micro-amplitude torsional frequency sweep to obtain the mechanical response; if the time-varying obstacle set B(t) If the risk exceeds the set threshold or the time window shrinks, the current action will be terminated and the user will wait for the next time window. Subsequently, the relative reflectance was obtained under annular, narrowband light source conditions. R 680 , R 730 , R 850 Obtain the chlorophyll degradation difference index S sp : ; Using reflectivity R 850 Brightness normalization and shadow compensation were performed, and the chlorophyll degradation difference index was calculated. S sp As the fruit ripens, the red blush on the surface tends to increase monotonically, which is used to suppress the interference of the red blush on the green base. Then, mechanical compliance is obtained, and the end effector of the harvesting robot arm is excited by angular displacement. By applying a small-amplitude torsional measurement torque and the signal value from the IMU, the frequency domain angular compliance can be obtained. : ; In the formula: Represents angular frequency The excitation amplitude; This represents the set of angular frequencies (in this embodiment, it is between 5 and 50). The frequency domain amplitude represents the angular displacement response; This represents the frequency domain amplitude of the torque excitation. Estimating the principal resonance frequency using a second-order equivalent model f r With damping ratio Thus, indicators of mechanical maturity are obtained. S mech : ; In the formula: These represent the weighting coefficients; This indicates the upper limit of the calibrated damping ratio; Mature fruits exhibit higher low-frequency compliance and lower resonant frequency / damping difference (i.e. S mech Increase); The main resonance frequency is estimated using a second-order equivalent model. f r With damping ratio Specifically, it includes: Output the angular displacement of the robotic arm's end effector under a small torsional excitation. With torque input Considering it as a linear second-order vibration system, its frequency domain transfer function is: ; In the formula: Indicates equivalent stiffness; , is the system's natural angular frequency; Indicates the equivalent damping ratio; Represents a constant. Angular compliance is measured. By analyzing the amplitude-frequency response within the resonance range, the peak frequency and half-power point can be determined, thereby estimating the principal resonant frequency. f r With damping ratio ; For example: Suppose that the maximum amplitude on the frequency response curve is... Appearance in frequency The frequency corresponding to the half-power point is ,satisfy The principal resonant frequency and damping ratio are respectively: ; To improve estimation accuracy, the discrete data obtained from the experiment can be processed. Curve fitting was performed using the least squares method.

[0041] Next, the microstructure optical parameters of the surface (such as cuticle density, waxiness, etc.) are processed: the harvesting robot uses a structured light sensor to image and obtain the width of the fruit's specular reflection lobes and the roughness of the fruit's microtexture, and then uses the full width at half maximum (FWHM) to analyze the data. w spec Constructing optical indices of fruit surface microstructure using speckle contrast K S surf : ; The maturation of the fruit's cuticle and wax layer leads to sharpening of the speckled lobes and a decrease in speckled contrast (i.e., optical indicators of the fruit's surface microstructure). S surf (Increase).

[0042] Furthermore, by reducing the disturbances caused by environmental factors through uncertainty, the degree of fruit ripeness can be assessed; specifically: The fruit maturity evidence function is obtained through nonlinear fusion. S mat : ; In the formula: These represent the corresponding weighting coefficients (in this embodiment, they are 0.45, 0.35, 0.2, and 0.25, respectively). These represent the indices obtained after normalization of the chlorophyll degradation difference index, the mechanical maturity index, and the optical index of the fruit surface microstructure, respectively; U represents the uncertainty measure. ; In the formula: u 1 represents the variance of repeated sampling, measured within a time window. The sample was repeated three times, and the ratio of standard deviation to mean was used to obtain the result. u 2 indicates dual-field-of-view consistency, determined by the maximum difference in separation time between BaseCam and EndCam (BaseCam is the base camera, EndCam is the recognition camera). If the maximum difference in separation time is no greater than 0.12s, the observation results of the two cameras are considered to be consistent in time. (For dual-field-of-view that are determined to be consistent, a unified field-of-view coordinate system is defined.) u 2. Perform feature fusion; otherwise, normalize by difference. u 3 indicates stability within the time window, i.e., the fluctuation range of the evidence function (maximum value - minimum value) / mean within the time window. ); For dual fields of view that are determined to be consistent, a unified field of view coordinate system is defined. u 2. Feature fusion is specifically performed as follows: Assume the pixel coordinates of BaseCam and EndCam in the current frame are respectively , ; when At that time, the weighted time average method was used to obtain the uniform field of view coordinates: ; Where: weight The determination is based on the spatial resolution and field-of-view overlap of the two cameras, respectively: ; In the formula: This represents the standard deviation of the imaging noise in the corresponding field of view. For empirical weighting factors; When the image quality of the two cameras is similar, one can choose... ,at this time u 2 is approximately the arithmetic mean of the coordinates of the two fields of view, i.e. ; If the time difference between the two fields of view exceeds 0.12 s, the data pair is considered asynchronous and will not participate in feature fusion of the current frame. Using the above method, spatially unified dual-field-of-view feature coordinates can be obtained while maintaining temporal consistency. u 2. Ensure the synchronization of global information of BaseCam and local features of EndCam in the spatiotemporal domain.

[0043] Final output maturity score With confidence level (Confidence level obtained from a large amount of experimental data): ; when and hour( This is the maturity threshold; in this embodiment, the value is 0.65. If the value is 0.6, the fruit is considered ripe; otherwise, the fruit is considered unripe or uncertain.

[0044] Step S3: Taking the optimal separation time as the target, and combining it with dynamic obstacle risk and end-effector alignment error, set the upper limit of the gripping force and the twisting speed of the picking claw, plan the smooth trajectory of the picking robotic arm, and maintain a safe distance between the two picking robotic arms to achieve fruit picking; during the picking process, monitor the force, posture, and risk threshold in real time, and if any abnormality is detected, immediately replan the picking process; specifically: First, generate the time-window gating trajectory: preset the initial pose. q(t 0 ) Picking posture is q pickThe trajectory of the right arm of the harvesting robot was parameterized using joint space splines: ; ; In the formula: a j The weighting coefficients represent the basis functions of the trajectory. Indicates the first j One time base function; n eef Indicates the direction of the screwing shaft at the end of the picking hand. n ped Indicates the normal direction of the fruit stalk; Indicates the angle between two directions; This indicates the alignment tolerance of the picking hand tip (in this embodiment, it is 8°). The weight coefficients of the trajectory basis function are obtained by minimizing the cost function of joint pose error and velocity smoothing constraint: ; In the formula: Indicates the smoothing constraint weight coefficient; Taken from the B-spline basis function family: ;in, P B The spline order is typically 3; the node vectors are determined based on the task time window. Divide into equal parts.

[0045] The basis functions satisfy: This ensures the continuity and differentiability of the trajectory.

[0046] a j and Together, they determine the joint space sampling trajectory of the robotic arm within the time window. Through the above parameterization, time-adjustable and posture-continuous trajectory control can be achieved. The comprehensive cost function is then obtained: ; In the formula: w t , w j , w r , w n These represent the corresponding weighting coefficients (in this embodiment, they are 3.0, 1.0, 2.0, and 1.5, respectively). This indicates the actual maximum separation time observed. X(t) Indicates the position of the picking claw tip; Subsequently, mechanical parameter feedforward and end-effector adaptive force control are performed: the set of mechanical parameters output in step S2 is used, including frequency domain angular compliance. Main resonant frequency f r With damping ratio This enables feedforward tuning of the gripper and selection control: The picking claw's gripping force is adaptive, and the maximum upper limit of the normal force is: ; In the formula: F base , F safe These represent the clamping reference force and the clamping safety force (in this embodiment, they are 10N and 3N, respectively). This represents a normalized mapping based on the training samples; ; The picking claw uses force-position control, and during the closing process... Contact compliance serves as feedback to avoid over-force. Harvesting claw twisting frequency matching: The twisting angular velocity and excitation frequency of the harvesting claw are determined by... Adaptive settings are used to ensure that the natural frequency of the fruit stem connection system is maintained, thereby further reducing the torque required for separation and the risk of the peel being sheared. Throughout the harvesting process, it is essential to ensure coordinated operation of both arms and a safe hand-switching domain. The left arm of the harvesting robot is designated for shielding, decoupling, and auxiliary support, while the right arm performs the harvesting. To prevent interference between the two arms, constraints on the safe distance and reachability of both arms are defined. ; In this embodiment, D min 0.15m represents the minimum distance between the two arms; In scenarios requiring a hand-changing position, introduce the hand-changing posture. q swap With the moment of change of hands t swap And apply derivative continuity: ; In the formula: , These represent the left and right limits at the moment of hand change, respectively; Regarding triggering, online monitoring, and exception handling, the triggering conditions are as follows: ; In the formula: t int This represents the risk score threshold (in this embodiment, the value is 0.15). And it satisfies the safety domain constraints of the two arms of the harvesting robotic arm; If the following abnormal states occur in the end-effector force-displacement or torque-angular displacement curves during the entire process, the harvesting robot arm will immediately release its grip and return to the pre-position, waiting for a replanning trigger or postponing to the next time window: (1) The slope of the torque-angular displacement curve exceeds the threshold: ; (2) Terminal posture deviation or ;in, This represents the end-effector pose deviation threshold (in this embodiment, the value is between 0.02 and 0.05 m). (3) Force control exceeding limits: ; (4) Risk Incentive Control: .

Claims

1. A dual-arm harvesting robot for spherical targets in a Gobi Desert environment, characterized in that: The system includes a collection chassis, a picking device, a sorting device, and a drive device. The collection chassis is a rectangular box structure with four cavities inside, separated by partitions, and six mounting columns evenly arranged on its bottom surface. Each of the six mounting columns has a drive device mounted at its bottom. Two sets of picking devices are mounted on the front top of the collection chassis via a motor-driven rotating platform, with a sorting device positioned between the two sets of picking devices. The motor-driven rotating platform is located on the top surface of the front cavity of the collection chassis, and the sorting devices correspond to the three cavities at the rear. A 3D LiDAR is installed at the front center of the motor-driven rotating platform of the collection chassis, and a control module, an IMU, and a battery module are installed in the cavity corresponding to the motor-driven rotating platform.

2. The dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 1, characterized in that: The harvesting device includes a lifting column, a base camera, a lifting screw, a lifting platform, a harvesting robotic arm, a harvesting claw, and a recognition camera. The lifting column is rotatably mounted on a motor rotating platform and has a hollow inner cavity with vertical sliding grooves on its four sides communicating with the inner cavity. The lifting screw is rotatably mounted inside the lifting column and has a base camera mounted on its top end face. The lifting platform is slidably mounted inside the lifting column and has its middle section threaded through by the lifting screw. The harvesting robotic arm is mounted on the outer wall of the front end of the lifting platform, and a harvesting claw is mounted on the end of the robotic arm away from the lifting platform. A recognition camera is mounted in the middle of the harvesting claw.

3. A dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 1 or 2, characterized in that: The harvesting robotic arm includes a mechanical forearm and a mechanical rear arm. One end of the mechanical forearm is connected to the lifting platform and the other end is connected to the mechanical rear arm. A harvesting claw is set at the end of the mechanical rear arm away from the mechanical forearm. Harmonic reducers are set at the connection points of the lifting platform, the mechanical forearm, and the mechanical rear arm.

4. A dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 1 or 3, characterized in that: The harvesting claw includes four claw mechanisms, a connecting turntable, a claw front shell, and a claw base. Each claw mechanism includes a claw, a first connecting rod, a second connecting rod, and a third connecting rod. The end of the claw is connected to the first connecting rod, and an anti-slip plate is provided on the inner side of the claw. A force sensor is installed between the anti-slip plate and the inner wall of the claw. The first and third connecting rods are connected via the second connecting rod, and the third connecting rod is connected to the connecting turntable. The connecting turntable is located in the claw front shell and controlled by a claw motor located in the claw front shell. A recognition camera is located in the middle of the end face of the claw front shell. The third connecting rod and... The central axis of the connection between the second link and the connecting turntable is parallel to the central axis of the front shell of the gripper. The central axis of the connection between the first link and the second link and the gripper is perpendicular to the central axis of the front shell of the gripper. The end face of the front shell of the gripper has a movable groove corresponding to the gripper. The side of the front shell of the gripper away from the gripper is rotatably connected to the gripper base, and a rotary motor is fixedly installed on the gripper base. The side of the gripper base away from the front shell of the gripper is located at the end of the harvesting robot arm. Both the front shell of the gripper and the gripper base are cylindrical. A six-axis torque sensor and an end effector IMU are installed between the harvesting gripper and the harvesting robot arm.

5. The dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 4, characterized in that: The sorting device includes a conical collector, a sorting tube, and a sorting rotary drum mechanism. The conical collector is made of elastic fabric, with its upper end expanding outward and its lower end connected to the sorting tube. The sorting tube is an L-shaped flexible tube, and its end is connected to the sorting rotary drum mechanism. The sorting rotary drum mechanism includes four sorting rings and twelve sorting rods. The four sorting rings are arranged in three sections corresponding to the three cavities at the rear end of the collection chassis, and the four sorting rings are coaxially arranged. The first section has five sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The second section has four sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The third section has three sorting rods, which are evenly distributed around the axis of the sorting rings on its outer ring. The angle of the line connecting the centers of the sorting rings is 10° upward.

6. The dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 5, characterized in that: The drive unit includes a main shaft, a main support frame, a drive shaft, a main drive wheel, auxiliary wheel side plates, auxiliary drive wheels, shock absorbers, hydraulic shock absorbers, and tracks. The main shaft is connected to a mounting column, and the main support frame is sleeved on the outer wall of the main shaft. The main support frame has an inverted T-shaped structure, with a drive shaft at its upper end and the main drive wheel sleeved on the outer wall of the drive shaft. Auxiliary wheel side plates are respectively set at both ends of the lower side of the main support frame. The auxiliary wheel side plates have a triangular structure, and auxiliary drive wheels and shock absorbers are respectively set at their other two ends. Hydraulic shock absorbers are set between each pair of the two shock absorbers and the main support frame, and the two shock absorbers have an inverted I-shaped structure. The tracks cover the outer walls of the main drive wheel and the two auxiliary drive wheels, and the tracks have a triangular structure.

7. The harvesting method of a dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 6, characterized in that: include: Step S1: First, extract row orientation and tree position based on 3DLiDAR, generate candidate observation poses and expand with small perturbation, and introduce occlusion sensitive field and picking window improvement index. Then, the overlap, incident tolerance, and predicted time-varying no-go region of the two picking robotic arms are calculated to obtain the optimal target pose and rolling replanning. Step S2: Within the allowable pose and time window obtained in step S1, first obtain spectral difference data, micro-torsional load compliance and fruit surface microstructure data, and construct a unified evidence function; then reduce the disturbance caused by environmental factors through uncertainty, thereby judging the degree of fruit ripeness. Step S3: Taking the optimal separation time as the target, and combining it with dynamic obstacle risk and end-effector alignment error, set the upper limit of the gripping force and the twisting speed of the picking claw, plan the smooth trajectory of the picking robotic arm, and at the same time maintain a safe distance between the two picking robotic arms to achieve fruit picking; during the picking process, monitor the force, posture and risk threshold in real time, and if an abnormality is found, immediately replan the picking process.

8. The harvesting method of a dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 7, characterized in that: The step S1, "extracting row orientation and tree position based on 3DLiDAR, generating candidate observation poses and expanding with small perturbations, and introducing occlusion-sensitive fields and harvesting window improvement indices," specifically includes: First, point cloud data of the fruit trees between rows was acquired using 3D LiDAR, and then the set of center points of the tree trunks was obtained through clustering and curve fitting. c i } and line baseline; Then, a set of candidate observation poses is generated in front of the harvesting robot using the baseline of the line: ； In the formula: p k Indicates planar position; Indicates the robot's orientation; To improve the accessibility and robustness of the harvesting robot, at each candidate observation pose... Based on this, a small perturbation is introduced to form an extended set. The final extended set obtained after constraints on the width of the inter-row passage and the turning radius of the picking robot is used as the input to the comprehensive cost function. And in each of the observed pose sets x k A limited sampling of lateral offset and orientation perturbations is set around the perimeter to form a local pose cluster to obtain an optimized search space; Next, the occlusion-sensitive potential field is calculated: first, within the field of view of each candidate observation pose, the fruit tree point cloud data obtained by the 3D LiDAR sensor is sliced ​​and its near-canopy projection duty cycle is calculated. d k With structural uncertainty Then, in coordination with the robot, it moved towards Orientation deviation Obtain occlusion risk indicators: ； ； In the formula: This represents the corresponding weighting coefficient; A occ (x k ) Represents the field of view x k Within the height range [ z min , z max The projected area of ​​the bottom surface covered by the point cloud; A fov (x k ) Represents the field of view x k The corresponding total visible ground area; Var z (x k ) Represents the field of view x k The variance of the corresponding point cloud slice in the depth direction; This represents the baseline standard deviation of the calibration scenario; Represents the field of view x k The angular deviation between the actual orientation and the reference orientation; wrap Indicates the angle wrapping operation; The range of observed poses to be analyzed is discretized to obtain a series of candidate poses. x k }, ensuring coverage of the spatial range of interest; for each candidate pose, obtain its occlusion risk index. Finally, the occlusion risk index of discrete candidate poses is extended to a continuous spatial field to obtain the occlusion-sensitive potential field. Next, by constructing a harvesting window improvement index, the maximum separation time of subsequent shading is predicted: ； In the formula: clip Represents the amplitude limiting function; This represents the time required to achieve maximum separation after the self-occlusion excitation is initiated under the reference pose condition; Indicates the forecast time window; D(x k ) This represents the distance parameter related to the observed pose; w k This represents the estimated value of the local crosswind. This indicates the angle between the camera and the fruit stand; These represent the corresponding weight coefficients; Improving the pose of indexed observations through the harvesting window x k The larger the index, the faster a stable harvestable window can be formed on the surface.

9. The harvesting method of a dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 8, characterized in that: The step S1, "calculating the overlap, incident tolerance, and predicted time-varying no-go region of the two picking robotic arms to obtain the optimal target pose and rolling replanning," specifically involves: First, set the left and right picking robotic arms in the observation position. x k The reachable areas are as follows: R L (x k ) , R R (x k ) Obtain the overlap of the workspace of the harvesting robotic arm: ； In the formula: area Indicates the area of ​​the region; Incident angle tolerance y k Used to represent the observed pose of the occlusion decoupling terminal. x k The effective angle of incidence for the picking target is determined; firstly, the angle of incidence of the picking claw tip to the picking fruit is obtained based on the current position of the picking claw, and then the angle of incidence tolerance is obtained through the feasible angle range for unobstructed picking. And an incident penalty factor is introduced. Pen inc : ； In the formula: y min The minimum tolerance threshold representing the angle of incidence; Next, the time-varying no-entry region is predicted: based on the shading-sensitive potential field, the branching model, and wind disturbance estimation, in the time domain... Internal prediction may cause time-varying obstacle sets for occlusion B(t) Introduce a soft constraint penalty term for collision risk: ； In the formula: This represents the minimum distance between the trajectory and the branch projection; Indicates the scale parameter; Indicates a time delay; Indicates the dynamic weighting coefficient; The prediction of time-varying forbidden regions is used to provide a safety margin for subsequent gating trajectories with respect to time windows; Finally, the comprehensive cost function is obtained based on the candidate observation pose set. J(x k ) : ； In the formula: w 1. w 2. w 3. w 4. w 5. w 6 represents the corresponding weight coefficients; Next navigation target location x * From the comprehensive cost function J(x k ) The decision was made to adopt a rolling time-domain optimization strategy, which means that the time-varying obstacle set is updated immediately after receiving a frame of data from the 3D LiDAR sensor and the IMU. B(t) With the maximum separation time And regain the comprehensive cost function. J(x k ) Finally, under the conditions of safety and the reachability of the picking robot, the next navigation path is obtained until the picking robot enters the predetermined observation pose point.

10. The harvesting method of a dual-arm harvesting robot for spherical targets in a Gobi Desert environment according to claim 9, characterized in that: In step S2, "within the allowable pose and time window obtained in step S1, first obtain spectral difference data, micro-torsional load compliance data, and fruit surface microstructure data, and then construct an evidence function in a unified manner," specifically: First, the observation pose given in step S1 x k With time window The spectral and surface imaging processes are completed internally, and the end effector of the right-side harvesting robotic arm does not exceed the preset safety amplitude. and normal force F max Under the condition of micro-amplitude torsional frequency sweep to obtain mechanical response; if time-varying obstacle set B(t) If the risk exceeds the set threshold or the time window shrinks, the current action will be terminated and the user will wait for the next time window. Subsequently, the relative reflectance was obtained under annular, narrowband light source conditions. R 680 , R 730 , R 850 Obtain the chlorophyll degradation difference index S sp : ； Using reflectivity R 850 Brightness normalization and shadow compensation were performed, and the chlorophyll degradation difference index was calculated. S sp As the fruit ripens, the red blush on the surface tends to increase monotonically, which is used to suppress the interference of the red blush on the green base. Then, mechanical compliance is obtained, and the end effector of the harvesting robot arm is excited by angular displacement. By applying a small-amplitude torsional measurement torque and the signal value from the IMU, the frequency domain angular compliance can be obtained. : ； In the formula: Represents angular frequency The excitation amplitude; Represents the set of angular frequencies; The frequency domain amplitude represents the angular displacement response; This represents the frequency domain amplitude of the torque excitation. Estimating the principal resonance frequency using a second-order equivalent model f r With damping ratio Thus, indicators of mechanical maturity are obtained. S mech : ； In the formula: These represent the weighting coefficients; This indicates the upper limit of the calibrated damping ratio; Mature fruits exhibit higher low-frequency compliance and lower resonant frequency / damping difference; Next, the microstructure optical parameters of the surface are processed: the harvesting robot uses a structured light sensor to image and obtain the width of the fruit's specular reflection lobes and the roughness of the fruit's microtexture, and then uses the full width at half maximum (FWHM) to analyze these parameters. w spec Constructing optical indices of fruit surface microstructure using speckle contrast K S surf : ； The maturation of the fruit's cuticle and wax layer leads to sharpening of the mirror petals and a reduction in speckle contrast.

Images