Ackerman mobile picking robot based on double-vision cooperation

Abstract

The invention is applicable to the technical field of agricultural automation equipment, and provides an Ackerman mobile picking robot based on double-vision collaboration, which comprises an Ackerman chassis, a six-degree-of-freedom mechanical arm, a power supply, a power supply and a control system, and further comprises a power supply system, a power supply system, a power supply system and a control system, and the six-degree-of-freedom mechanical arm is provided with a shear type end effector; the global vision module is mounted at the front end of the Ackerman chassis through a two-dimensional holder; the hand-eye vision module is fixedly mounted on the shearing type end effector; and the control unit is integrated in the industrial personal computer. The device has both efficiency and precision, has strong error compensation capability, is reasonable in structure and good in adaptability, and has strong anti-interference capability.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of agricultural automation equipment, and particularly relates to an Ackermann mobile picking robot based on double-vision cooperation. BACKGROUND

[0002] Fruit picking is an important link in agricultural production, and at present, most orchards still rely on manual picking, which has problems of high labor intensity, low efficiency, high cost and the like. With the development of automation technology, picking robots have become a research hotspot.

[0003] The existing picking robot technical solutions mainly have the following deficiencies: first, the robot with a fixed base lacks mobility and has a limited working range. Second, the robot with a mobile platform mostly uses a single vision sensor. If only a global camera is used, the image is prone to shaking when the vehicle moves because the global camera is fixed on the mobile chassis, and the long-distance recognition and positioning accuracy is low, which cannot meet the requirement of accurate picking; if only an eye-in-hand camera is used, the mechanical arm needs to perform a large range of blind scanning to find the target, and the search efficiency is extremely low. Third, many mobile platforms use a differential chassis structure, and the kinematic model and coordinated control of the mechanical arm are relatively simple, but in the complex orchard terrain, the Ackermann chassis has an advantage in terms of turning structure and better stability. However, the non-omnidirectional motion characteristics of the Ackermann chassis will cause the parking pose to have lateral and heading deviations, and the traditional control method is difficult to enable the mechanical arm to directly perform accurate positioning based on the vision information before moving, and currently there is a lack of effective solutions to compensate for the error introduced by the specific motion model.

[0004] Therefore, there is an urgent need for a picking robot that can balance wide-range and efficient search, high-precision positioning picking, and effectively adapt to unstructured orchard terrain and compensate for the movement deviation. SUMMARY

[0005] The problems in the background art are solved.

[0006] The embodiment of the application is implemented as follows: an Ackermann mobile picking robot based on double-vision cooperation, comprising an Ackermann chassis, and further comprising: a six-degree-of-freedom mechanical arm, a base of the six-degree-of-freedom mechanical arm being fixedly installed on a vehicle frame of the Ackermann chassis, and a shearing type end effector being arranged on the six-degree-of-freedom mechanical arm, the shearing type end effector being used for grabbing fruits and shearing off fruit stems; a global vision module, the global vision module being installed at a front end of the Ackermann chassis through a two-dimensional holder, the global vision module being used for collecting color images and depth information of the front environment, and the two-dimensional holder being used for keeping the posture of the global vision module stable; A hand-eye vision module is fixedly installed on a shearing end effector of the six-degree-of-freedom robot arm, and is used for collecting images of target fruits at a close distance; A control unit is integrated in the industrial computer, and is in communication connection with the controller of the Ackerman chassis, the controller of the six-degree-of-freedom robot arm, the controller of the two-dimensional holder, the global vision module and the hand-eye vision module, respectively, and is used for receiving data collected by each module and performing control through closed-loop feedback.

[0007] In a further technical solution, the Ackerman chassis adopts a four-wheel structure, the front wheels are steering wheels, the rear wheels are driving wheels, and the Ackerman chassis is powered by a lithium battery pack.

[0008] In a further technical solution, the shearing end effector is an electric shearing device, has a jaw structure and a shearing structure, and is provided with a built-in pressure sensor, and is used for grabbing fruits and shearing off fruit stems.

[0009] In a further technical solution, the two-dimensional holder is fixedly installed at the front end of the Ackerman chassis through a support, has a horizontal rotation angle range of 0-360° and a pitch angle range of -30° to +90°, and is provided with an IMU (inertial measurement unit) sensor and an attitude sensor inside, and is used for monitoring real-time changes in attitude angles thereof.

[0010] In a further technical solution, the industrial computer is installed in a waterproof electrical box on the Ackerman chassis.

[0011] In a further technical solution, the control method of the Ackerman mobile picking robot comprises the following steps: Step 1: global coarse positioning and chassis guidance; After the system is initialized, the global vision module is driven by the two-dimensional holder to scan the working area; during movement of the Ackerman chassis, the industrial computer continuously reads IMU sensor data of the Ackerman chassis and the two-dimensional holder, dynamically adjusts a rudder of the two-dimensional holder through a PID control algorithm, compensates for shaking of the Ackerman chassis, maintains a stable observation posture of the global vision module, continuously collects images, identifies fruit targets through an image recognition algorithm, and calculates a first coordinate P_global of the target fruit relative to a coordinate system {O_global} of the global vision module; Step 2: coordinate conversion and path planning; The industrial computer converts P_global from the {O_global} coordinate system directly to the coordinate system {O_base} of the six-degree-of-freedom robot arm according to the conversion matrix obtained through calibration in advance, to obtain a second coordinate P_approx for the pre-positioning of the six-degree-of-freedom robot arm; at the same time, P_global is converted to the coordinate system {O_akm} of the Ackermann chassis to obtain a third coordinate P_akm, and then the target pose (X_target, Y_target, θ_target) to be moved by the Ackermann chassis is calculated by taking the third coordinate P_akm as input; at the same time, the controller of the six-degree-of-freedom robot arm receives the P_approx coordinate, calculates the target pose (X_gripper, Y_gripper, θ_gripper) of the shearing end effector, and controls the six-degree-of-freedom robot arm to start the pre-positioning movement towards the target fruit; Step 3: chassis movement; The Ackermann chassis receives the instruction and moves to the target pose; Step 4: hand-eye fine calibration; The six-degree-of-freedom robot arm is controlled to move, the hand-eye vision module immediately captures a high-definition image, the image processing algorithm is used to accurately position the fruit, and the fourth coordinate P_precise of the fruit relative to the base coordinate system {O_base} of the robot arm is calculated in combination with the known hand-eye relationship, so as to correct all the accumulated errors in the previous stage, and the precise calibration of the pre-positioning coordinate P_approx of the six-degree-of-freedom robot arm is realized through closed-loop feedback, and the target precise pose (X_gripper_approx, Y_gripper_approx, θ_gripper_approx) to be moved by the shearing end effector is calculated; Step 5: picking execution; The six-degree-of-freedom robot arm plans a collision-free path from (X_gripper, Y_gripper, θ_gripper) to (X_gripper_approx, Y_gripper_approx, θ_gripper_approx), and the shearing end effector completes the shearing action; Step 6: cycle decision; After picking is completed, the control unit decides whether to continue picking other fruits of the current plant or to control the Ackermann chassis to move to the next global target point.

[0012] The Ackermann mobile picking robot based on double-vision cooperation provided by the embodiment of the application has the following beneficial effects: (1) Efficiency and accuracy: through the global vision module for large-scale, efficient target search and rough positioning, guide the Ackerman chassis movement; through the hand-eye vision module in close proximity to the final positioning of millimeter level accuracy, guide the picking action, perfect solution to the "search" and "positioning" contradiction.

[0013] (2) Strong error compensation ability: creatively the hand-eye vision module precision positioning process as the whole system closed loop feedback, can effectively compensate for the Ackerman chassis due to non-omnidirectional motion model of parking pose error, vision system calibration error and other random interference in the orchard environment, greatly improve the robustness and picking success rate of the system.

[0014] (3) Reasonable structure, good adaptability: the Ackerman chassis is more suitable for outdoor uneven terrain movement, the arrangement of the double vision module gives full play to their respective advantages, the whole system architecture is optimized for modern agricultural application scenarios, strong practicality.

[0015] (4) Strong anti-interference ability: by introducing a two-dimensional gimbal based on PID algorithm, effectively suppresses the shaking and impact of the Ackerman chassis when moving on unstructured terrain, ensures that stable images can be obtained during movement, improves the overall robustness of the system in complex environments. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A structure schematic diagram of an Ackerman mobile picking robot based on double vision cooperation is provided for the embodiments of the present application. Figure 2 A side view of an Ackerman mobile picking robot based on double vision cooperation is provided for the embodiments of the present application. Figure 3 A top view of an Ackerman mobile picking robot based on double vision cooperation is provided for the embodiments of the present application. Figure 4 A coordinate conversion relationship schematic diagram of an Ackerman mobile picking robot based on double vision cooperation is provided for the embodiments of the present application. Figure 5 A work flow diagram of a double vision cooperation control method of an Ackerman mobile picking robot based on double vision cooperation is provided for the embodiments of the present application.

[0017] In the drawings: Ackerman chassis 1; six degrees of freedom robot arm 2; hand-eye vision module 3; shearing type end effector 4; global vision module 5; two-dimensional gimbal 6; industrial computer 7. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0019] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0020] like Figures 1-3 As shown, an Ackerman mobile harvesting robot based on dual-vision collaboration is provided in one embodiment of the present invention, including an Ackerman chassis 1, and further comprising: A six-degree-of-freedom robotic arm 2, the base of which is fixedly mounted on the frame of the Ackerman chassis 1, is provided with a shearing end effector 4, which is used to grasp fruit and cut off the fruit stem; The global vision module 5 is mounted on the front end of the Ackerman chassis 1 via a two-dimensional gimbal 6. The global vision module 5 is used to acquire color images and depth information of the environment in front, and the two-dimensional gimbal 6 is used to maintain the attitude stability of the global vision module 5. Hand-eye vision module 3, which is fixedly installed on the shearing end effector 4 of the six-degree-of-freedom robotic arm 2, is used to acquire images of the target fruit at close range; The control unit, which is integrated into the industrial computer 7, is communicatively connected to the controller of the Ackerman chassis 1, the controller of the six-degree-of-freedom robotic arm 2, the controller of the two-dimensional gimbal 6, the global vision module 5, and the hand-eye vision module 3, respectively. It is used to receive data collected by each module and perform control through closed-loop feedback.

[0021] In this embodiment of the invention, the control unit dynamically adjusts the two-dimensional gimbal 6 according to the data from the attitude sensor and inertial measurement unit sensor built into the two-dimensional gimbal 6 through a PID control algorithm to maintain the attitude stability of the global vision module 5 when the Ackerman chassis 1 moves and shakes; the global vision module 5 is a vision sensor with depth perception function, and its depth vision field angle is greater than that of the hand-eye vision module 3 to achieve a wider range of target search.

[0022] In a preferred embodiment of the present invention, the Ackerman chassis 1 adopts a four-wheel structure, with the front wheels being steering wheels and the rear wheels being drive wheels, powered by a lithium battery pack.

[0023] In a preferred embodiment of the present invention, the hand-eye vision module 3 is fixed above the shear-type end effector 4 by a lightweight bracket, and its optical axis direction is at a certain angle to the working direction of the shear-type end effector 4 to avoid obstruction.

[0024] In a preferred embodiment of the present invention, the shearing end effector 4 is an electric shearing device with a gripper structure and a shearing structure, and a built-in pressure sensor for gripping the fruit and cutting the fruit stem.

[0025] In a preferred embodiment of the present invention, the two-dimensional gimbal 6 is fixedly mounted on the front end of the Ackerman chassis 1 via a bracket, and its horizontal rotation angle range is 0-360°, and its pitch angle range is -30° to +90°. The two-dimensional gimbal 6 has a built-in IMU (Inertial Measurement Unit) sensor and an attitude sensor for real-time monitoring of its own attitude angle changes.

[0026] In a preferred embodiment of the present invention, the industrial control computer 7 is installed in a waterproof electrical box on the Ackerman chassis 1.

[0027] like Figure 5 As shown, in a preferred embodiment of the present invention, the control method of the Ackerman mobile harvesting robot includes the following steps: Step 1: Global coarse positioning and chassis guidance; After system initialization, the global vision module 5, driven by the 2D gimbal 6, scans the work area. During the movement of the Ackerman chassis 1, the industrial control computer 7 continuously reads the IMU sensor data from the Ackerman chassis 1 and the 2D gimbal 6. It dynamically adjusts the servo motor of the 2D gimbal 6 using a PID control algorithm to compensate for the shaking of the Ackerman chassis 1 and maintain the stable observation posture of the global vision module 5. The industrial control computer 7 runs a target detection algorithm (all algorithms used are existing technologies and will not be specifically described here) to identify ripe fruit in the image. Using the intrinsic parameters and pre-calibrated extrinsic parameters of the global vision module 5, and through the principle of visual ranging, it calculates the three-dimensional coordinates P_global (the first coordinate) of the fruit target in the coordinate system {O_global} of the global vision module 5.

[0028] Step 2: Coordinate transformation and path planning; The industrial control computer 7, based on the transformation matrix obtained through pre-calibration, directly transforms P_global from the {O_global} coordinate system to the {O_base} coordinate system of the six-degree-of-freedom robotic arm 2, obtaining the coordinate P_approx (second coordinate) for the pre-positioning of the six-degree-of-freedom robotic arm 2. The system initiates a parallel processing mechanism: transforming P_global to the coordinate system {O_akm} of the Ackerman chassis 1, obtaining the coordinate P_akm (third coordinate). Subsequently, the path planning module uses P_akm as input to calculate the target pose (X_target, Y_target, θ_target) that the Ackerman chassis 1 needs to move. This pose ensures that the fruit is located within the optimal area of ​​the six-degree-of-freedom robotic arm 2's workspace. Simultaneously, the controller of the six-degree-of-freedom robotic arm 2 receives the P_approx coordinate and calculates the target pose (X_gripper, Y_gripper, θ_gripper) of the shearing end effector 4, controlling the six-degree-of-freedom robotic arm 2 to begin pre-positioning movement towards the approximate location of the target fruit, significantly shortening the time for subsequent fine positioning.

[0029] Step 3: Chassis movement; Ackerman chassis 1 receives the command and moves to the target pose.

[0030] Step 4: Hand-eye calibration; Due to uneven ground and motion control errors, the actual parking posture of the Ackerman chassis 1 deviates from the target posture. At this time, the six-DOF robotic arm 2 is already in the approximate target area. The hand-eye vision module 3 immediately captures a high-definition image and uses a sophisticated image processing algorithm to accurately locate the fruit. Combined with the known hand-eye relationship (obtained through hand-eye calibration), the precise coordinates P_precise (the fourth coordinate) of the fruit relative to the robotic arm's base coordinate system {O_base} are calculated. This coordinate P_precise corrects all previously accumulated errors and, through closed-loop feedback, accurately corrects the pre-positioned coordinates P_approx of the six-DOF robotic arm 2, calculating the precise target posture (X_gripper_approx, Y_gripper_approx, θ_gripper_approx) of the shearing end effector 4.

[0031] Step 5: Harvesting execution; The six-degree-of-freedom robotic arm 2 plans a collision-free path to move from (X_gripper, Y_gripper, θ_gripper) to (X_gripper_approx, Y_gripper_approx, θ_gripper_approx), the shearing end effector 4 completes the shearing action, and the sensor confirms that the harvesting was successful.

[0032] Step 6: Iterative decision-making; After harvesting is completed, the control unit decides whether to continue harvesting other fruits from the current plant or to control the Ackerman chassis 1 to move to the next global target point.

[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An Ackerman mobile harvesting robot based on dual-vision collaboration, characterized in that, Including the Ackerman chassis, it also includes: A six-degree-of-freedom robotic arm, the base of which is fixedly mounted on the frame of the Ackerman chassis, is equipped with a shearing end effector for grasping fruit and cutting off the fruit stem; A global vision module is mounted on the front end of the Ackerman chassis via a two-dimensional gimbal. The global vision module is used to acquire color images and depth information of the environment in front, and the two-dimensional gimbal is used to maintain the attitude stability of the global vision module. The hand-eye vision module is fixedly installed on the shearing end effector of the six-degree-of-freedom robotic arm and is used to acquire images of target fruits at close range. The control unit, integrated within the industrial computer, is communicatively connected to the controllers of the Ackerman chassis, the six-degree-of-freedom robotic arm, the two-dimensional gimbal, the global vision module, and the hand-eye vision module. It receives data collected by each module and performs control through closed-loop feedback.

2. The Ackerman mobile harvesting robot based on dual-vision collaboration according to claim 1, characterized in that, The Ackerman chassis has a four-wheel structure, with the front wheels being steering wheels and the rear wheels being drive wheels, powered by a lithium battery pack.

3. The Ackerman mobile harvesting robot based on dual-vision collaboration according to claim 1, characterized in that, The shearing end effector is an electric shearing device with a gripper structure and a shearing structure, and a built-in pressure sensor, used to grasp the fruit and cut the fruit stem.

4. The Ackerman mobile harvesting robot based on dual-vision collaboration according to claim 1, characterized in that, The two-dimensional gimbal is fixedly installed at the front end of the Ackerman chassis by a bracket, and its horizontal rotation angle range is 0-360°, and its pitch angle range is -30° to +90°. The two-dimensional gimbal has a built-in IMU sensor and attitude sensor for real-time monitoring of its own attitude angle changes.

5. The Ackerman mobile harvesting robot based on dual-vision collaboration according to claim 1, characterized in that, The industrial control computer is installed in a waterproof electrical box on an Ackerman chassis.

6. The Ackerman mobile harvesting robot based on dual-vision collaboration according to claim 4, characterized in that, The control method for the Ackerman mobile harvesting robot includes the following steps: Step 1: Global coarse positioning and chassis guidance; After system initialization, the global vision module scans the work area under the drive of the 2D gimbal. During the movement of the Ackerman chassis, the industrial control computer continuously reads the IMU sensor data of the Ackerman chassis and the 2D gimbal, and dynamically adjusts the servo motor of the 2D gimbal through the PID control algorithm to compensate for the shaking of the Ackerman chassis and maintain the stable observation posture of the global vision module. The global vision module continuously acquires images, identifies the fruit target through the image recognition algorithm, and calculates the first coordinate P_global of the target fruit relative to the coordinate system {O_global} of the global vision module. Step 2: Coordinate transformation and path planning; The industrial control computer, based on the transformation matrix obtained through pre-calibration, directly transforms P_global from the {O_global} coordinate system to the coordinate system {O_base} of the six-degree-of-freedom robot arm, obtaining the second coordinate P_approx used for the pre-positioning of the six-degree-of-freedom robot arm; simultaneously, it transforms P_global to the coordinate system {O_akm} of the Ackerman chassis, obtaining the third coordinate P_akm. Then, using the third coordinate P_akm as input, it calculates the target pose (X_target, Y_target, θ_target) that the Ackerman chassis needs to move; at the same time, the controller of the six-degree-of-freedom robot arm receives the P_approx coordinates, calculates the target pose (X_gripper, Y_gripper, θ_gripper) of the shearing end effector, and controls the six-degree-of-freedom robot arm to begin the pre-positioning movement towards the target fruit; Step 3: Chassis movement; The Ackermann chassis receives commands and moves to the target pose; Step 4: Hand-eye calibration; Controlling the movement of the six-degree-of-freedom robotic arm, the hand-eye vision module immediately captures high-definition images. Image processing algorithms are used to accurately locate the fruit, and combined with the known hand-eye relationship, the fourth coordinate P_precise of the fruit relative to the robotic arm's base coordinate system {O_base} is calculated, thereby correcting all previously accumulated errors. Through closed-loop feedback, the coordinate P_approx of the six-degree-of-freedom robotic arm's pre-positioning is accurately corrected, and the precise pose of the target (X_gripper_approx, Y_gripper_approx, θ_gripper_approx) of the shearing end effector is calculated. Step 5: Harvesting execution; A six-DOF robotic arm plans a collision-free path from (X_gripper, Y_gripper, θ_gripper) to (X_gripper_approx, Y_gripper_approx, θ_gripper_approx), and a shearing end effector completes the shearing action. Step 6: Iterative decision-making; After harvesting, the control unit decides whether to continue harvesting other fruits from the current plant or to control the Ackerman chassis to move to the next global target point.

Images