Multi-view collaborative multi-mechanical-arm autonomous walking grape picking robot and control method thereof
Through a multi-mechanical multi-arm autonomous walking grape picking robot, combined with navigation magnetic stripes and fiber optic sensors, the precise picking and efficient collection of grapes are achieved, solving the problems of low picking success rate and high damage rate in the existing technology, and achieving Low-cost self-guided navigation continuous operation and efficient picking effect.
Patent Information
- Application Number
- CN202411432163.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-14
- Publication Date
- 2024-11-26
AI Technical Summary
The existing grape picking robots have problems such as difficulty in identifying picking points, low picking success rate, low harvesting efficiency and high damage rate. They also have high manufacturing cost, difficulty in controlling multiple degrees of freedom and low accuracy, so they cannot achieve low-cost self-propelled navigation continuous Operation.
Using multi-mesh synergistic multi-robot autonomous walking grape picking robot, combining navigation magnetic stripes, AGV magnetic navigation sensors, global depth cameras, robotic arm depth cameras, shear clamping mechanical claws and frame matrix fiber sensors, through navigation magnetic stripes and The AGV magnetic navigation sensor realizes route navigation, coordinated positioning of the global depth camera and the robotic arm depth camera, shear clamping mechanical claws for precise picking, and optimized collection and placement through the frame matrix fiber sensor.
Improve the picking success rate and overall efficiency, reduce the damage rate, realize low-cost self-guided continuous operation, reduce manufacturing costs, improve positioning accuracy and control accuracy, reduce motion interference between robotic arms and fruit particles Shedding and peel abrasion.
Smart Images

Figure CN119014223A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of agricultural harvesting machinery, and in particular to a multi-eye coordinated multi-mechanical arm autonomous walking grape picking robot and a control method thereof. Background Art
[0002] At present, my country has become the world's largest producer and consumer of fresh table grapes. This type of grape has problems such as short harvest period and high labor intensity. Therefore, grape picking robots are of great significance for realizing automated and intelligent grape picking. They can promote the intelligentization process of vineyards and improve the production efficiency and economic benefits of vineyards.
[0003] The existing trellis-cultivated table grape picking robots mainly have the problems of difficult picking point identification, low picking success rate, low harvesting efficiency, and high damage rate. The research on grape harvesting technology is mostly focused on grape identification and positioning. At the same time, most picking robot arms are industrialized and commercial multi-joint robot arms, which have the problems of high manufacturing cost, high degree of freedom, difficult control, and low precision. In addition, the existing picking robots are limited by the high cost of GPS navigation and the poor signal in the facility, and cannot achieve continuous operation of self-propelled navigation, which greatly affects the operation efficiency of the machine.
[0004] Therefore, there is an urgent need to study continuous harvesting equipment that can improve the picking success rate, improve overall efficiency and reduce the damage rate at low cost, so as to solve the bottleneck problem of automation and intelligent operation of fresh grapes and dried grapes cultivated on trellis. Summary of the invention
[0005] In view of the defects existing in the prior art, the purpose of the present invention is to provide a multi-coordinated multi-arm autonomous grape picking robot and a control method thereof. The device can improve the success rate of picking, improve the overall efficiency and reduce the damage rate while controlling the cost, and can also realize self-propelled navigation and continuous operation.
[0006] In order to achieve the above purpose, the technical solution adopted by the present invention is:
[0007] A multi-camera coordinated multi-mechanical arm autonomous walking grape picking robot, including a navigation magnetic strip, a lifting crawler chassis, a control cabinet, a coordinate mechanical arm, a shearing and clamping mechanical claw, an AGV magnetic navigation sensor, a collection device, a global depth camera and a mechanical arm depth camera;
[0008] The navigation magnetic strip is laid on the ground according to the designed picking track, and the AGV magnetic navigation sensor is fixedly installed on the right side of the crawler chassis body. The navigation magnetic strip cooperates with the AGV magnetic navigation sensor to realize the route navigation of the machine;
[0009] The global depth camera is fixedly mounted on the rightmost side of the upper plane of the lifting crawler chassis and is used to identify grapes and assign picking tasks;
[0010] The coordinate robot arm is fixedly installed above the lifting crawler chassis, and a shearing clamping mechanical claw and a robot arm depth camera are fixedly installed at the end of the coordinate robot arm. The robot arm depth camera cooperates with the global depth camera to realize the coordinated positioning of grapes;
[0011] The collecting device is placed under the shearing and clamping mechanical claws to collect grapes and guide the coordinate mechanical arm to place the grapes.
[0012] On the basis of the above scheme, the lifting chassis is an electric crawler chassis, which mainly includes two sets of crawler groups, a crawler chassis body, an electric push rod, a lifting support rod and a mounting platform. Two sets of track wheel groups are installed on both sides of the crawler chassis body. The two ends of the lifting support rod are respectively installed on the crawler chassis body and the mounting platform by means of hinge connection and slider connection. The two ends of the electric push rod are respectively installed on the crawler chassis body and the sliding end of the lifting support rod. The extension and retraction of the electric push rod can realize the lowering and raising of the mounting platform, and the lifting function can enable the machine to adapt to different planting heights.
[0013] On the basis of the above scheme,
[0014] There are a total of 3 groups of coordinate robotic arms, which are arranged horizontally. The Y-axis screw module of the coordinate robotic arm is fixed to the mounting platform by a bolt group, the Z-axis electric cylinder module is fixed to the Y-axis slide by screws, the X-axis screw module is fixed to the Z-axis mounting plate by screws, the mechanical claw mounting rod is fixed to the X-axis slide by buckles and screws, and the shear clamping mechanical claw is installed at the end of the mechanical claw mounting rod by screws and installed on the mounting platform, located in front of the collection device.
[0015] On the basis of the above scheme, the shearing and clamping mechanical claw is fixedly installed with a servo and a claw guide rail using a mechanical claw bracket as a carrier, a steering wheel is fixedly connected to the output shaft of the servo, two connecting rods are connected to the steering wheel through a hinge, and the two connecting rods are respectively connected to the shearing finger and the supporting finger through a hinge, and the shearing finger is slidably installed on the claw guide rail through a slider S, and the supporting finger is slidably installed on the claw guide rail through a slider B; the rotational motion of the servo is transmitted through the steering wheel and the connecting rod, and is converted into the opposite linear motion of the shearing finger and the supporting finger along the claw guide rail, thereby realizing the shearing and clamping action;
[0016] The shear finger uses a shear finger bracket as a bracket, and its rear end is fixedly connected to a connecting rod S and a shear finger slider by screws, and a shear guide strip, a pad and a claw matrix optical fiber sensor transmitting end are fixedly installed on the upper plane in sequence, and a blade is fixedly installed in a blade installation groove on its side, and a clamping block S is assembled in a corresponding square hole on the shear finger bracket in the form of a slider, and a rivet S is fixedly installed on the clamping block S, and a spring S is sleeved on the rivet S, and both ends of the spring S are in contact with the clamping block S and the shear finger bracket respectively;
[0017] The supporting finger uses the supporting finger bracket as a bracket, and its rear end is fixedly connected to the connecting rod B and the supporting finger slider by screws. The pad, supporting guide strip and claw matrix optical fiber sensor receiving end are fixedly installed on the upper plane in sequence. The clamping block B is assembled in the corresponding square hole on the supporting finger bracket in the form of a slider. The clamping block B is also fixedly installed with a rivet B, and a spring B is sleeved on the rivet B. The two ends of the spring B are respectively in contact with the clamping block B and the shear finger bracket.
[0018] On the basis of the above scheme, the shearing guide strip and the supporting guide strip form a trapezoidal guide area that gradually opens outwards and forms a closed rectangular shearing area inwards. The guide area guides the grape stalks into the shearing area to wait for shearing and clamping by the shearing clamping mechanical claws. At the same time, the claw matrix optical fiber sensor receiving end cooperates with the claw matrix optical fiber sensor receiving end to determine whether there are stalks in the shearing area, thereby realizing the inspection of whether the coordinate robot arm is accurately in place. Due to the existence of the guide area, the requirements for the movement accuracy of the coordinate robot arm and the positioning accuracy of the camera on the picking point can be greatly reduced;
[0019] When the shearing and holding mechanical claw is in action, the shearing fingers and the supporting fingers move toward each other along the claw guide rail, the clamping block S and the clamping block B contact the fruit stalk, the spring S and the spring B are gradually compressed, and a holding force is applied to the two clamping blocks. The clamping force acts on the fruit stalk, clamps the fruit stalk to complete the holding action, and at the same time, the blade gradually approaches and is embedded in the knife groove of the supporting finger bracket to complete the shearing action.
[0020] On the basis of the above scheme, the collection box of the collection device is placed in the limit groove of the installation platform, below the shearing and clamping mechanical claw, and the frame matrix optical fiber sensor transmitting end and the frame matrix optical fiber sensor receiving end are respectively installed on both sides of the collection box through sensor brackets, and both are slightly higher than the upper frame edge of the collection box. The frame matrix optical fiber sensor detects the distribution of grape bunches on the upper edge of the collection box, guides the coordinate robot arm to place the grape bunches in the collection box, and ensures that the collection box is evenly filled; at the same time, it can detect whether the collection box is full, which is used to guide whether the frame needs to be replaced.
[0021] On the other hand, the present invention also provides a control method for a multi-coordinated multi-mechanical arm autonomous walking grape picking robot, comprising the following steps:
[0022] Step 1. According to the planting pattern and layout of the orchard, the working route is planned in combination with the turning radius and working width of the machine, and the navigation magnetic strip is laid on the working route. The magnetic strip can be used many times after being laid once, and then the electric push rod is adjusted according to the working height to make the machine reach a suitable working height; during the operation, the lifting crawler chassis moves forward along the navigation magnetic strip with the X-axis travel of the robot arm as the unit length under the guidance of the AGV magnetic navigation sensor and the path detection system, and enters the next unit length for operation after picking a total unit length.
[0023] Step 2: The global depth camera collects image information and depth information above it, and transmits the information to the host computer located in the control box through the serial port. The host computer processes the received information based on a deep learning algorithm including but not limited to yolov8, and after loading the model parameters and weights, inputs the preprocessed image into the YOLO model; performs forward propagation, and detects targets on feature maps of different scales through multiple convolution and pooling layers; performs target detection at each scale, and predicts multiple bounding boxes and corresponding category probabilities of each grid through the convolution layer; decodes the prediction of each bounding box to obtain the center coordinates, width, height and confidence score of the bounding box, and then calibrates the position of the grape bunch, and then determines the point above the center point of the upper edge of the bounding box at a certain distance as the picking point, and determines the coordinates of the picking point relative to the global depth camera in combination with the depth information;
[0024] Step 3, assigning a coordinate robot arm to each bunch of grapes according to the distribution of the grape bunches processed in step 2. By rationally planning the order of picking the grape bunches and the motion trajectory of each robot arm, the motion interference between the robots is avoided.
[0025] Step 4, according to the picking tasks assigned in step 3 and the picking order and picking trajectory determined, each group of coordinate robotic arms starts to pick grape bunches in turn. Each picking is divided into two stages. In the first stage, the Y-axis and Z-axis coordinates of the shearing and clamping mechanical claws are controlled to be roughly aligned. In the X-axis direction, the shearing and clamping mechanical claws are located a certain distance behind the grape bunches; in the second stage, the depth camera performs a secondary detection on the grape bunches to determine their relative coordinates. Since the distance between the robotic arm depth camera and the grape bunches is relatively close at this time, the error of the determined coordinates is small, which can greatly improve the positioning accuracy; after the secondary positioning is completed, the robotic arm is controlled to move secondary, and the grape stalks are guided into the shearing area by the guide area, and then sheared after being determined by the claw matrix fiber optic sensor. If the claw matrix fiber optic sensor confirms failure, the robotic claw returns to the starting point of the second stage, and picks after another detection until it is confirmed that the picking is completed;
[0026] Step 5, after cutting and clamping the grape bunches, the placement action is performed, the matrix fiber optic sensor, the frame matrix fiber optic sensor transmitting end and the matrix fiber optic sensor receiving end detect the distribution of the grape bunches on the upper edge of the collection box, and guide the coordinate robot arm to place the grape bunches in the collection box to ensure that the collection box is evenly filled; when it is detected that the grape bunches on the upper edge of the collection box are full, the movement is stopped and the collection box is replaced; when placing, the robot arm fills the collection box layer by layer in an orderly manner according to the prescribed placement order, and gradually lowers the height of the grape bunch in the process. When the bottom of the grape bunch contacts the bottom of the collection box or other grape bunches, the grape bunch changes from downward movement to oblique downward movement, and gradually transitions from a vertical state to a horizontal state to prevent the grape berries from falling off and the skin from being scratched due to collision.
[0027] The beneficial effects of the present invention are:
[0028] (1) With the help of multi-eye collaborative positioning solution, the positioning accuracy is improved. The picking process is divided into two stages. The secondary positioning distance in the second stage is short and the positioning accuracy is high.
[0029] (2) The guided shearing and clamping mechanical claw combined with the matrix fiber optic sensor reduces the dependence on positioning accuracy and control accuracy, ensuring the reliability and credibility of the shearing action. The ingenious structure of the shearing and clamping mechanical claw only uses one servo to complete the two actions of shearing and clamping, reducing the control difficulty and manufacturing cost.
[0030] (3) Multiple coordinate manipulators work simultaneously, greatly improving the overall efficiency of the operation. At the same time, the use of coordinate manipulators also solves the problem of the high cost of multi-degree-of-freedom joint robots.
[0031] (4) With the help of navigation magnetic strips and AGV magnetic navigation sensors, the high cost of GPS navigation solutions and poor signal conditions within the facility are circumvented, and the function of self-propelled navigation and continuous operation is achieved at a low cost, ensuring operational efficiency.
[0032] (5) The coordinate robot arm is combined with the frame matrix fiber optic sensor to evenly fill the grape collection box layer by layer. At the same time, the reasonable placement action of the coordinate robot arm can reduce the loss of particles and the abrasion of the skin. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The present invention has the following accompanying drawings:
[0034] Figure 1 , Figure 2 It is an overall schematic diagram of the device described in this patent;
[0035] Figure 3 It is a front view of the device described in this patent;
[0036] Figure 4 is a top view of the device described in this patent;
[0037] Figure 5 It is a left side view of the device described in this patent;
[0038] Figure 6 It is an overall schematic diagram of the lifting crawler chassis of the device described in this patent;
[0039] Figure 7 It is an overall schematic diagram of the coordinate robot arm of the device described in this patent;
[0040] Figure 8 It is an overall schematic diagram of the shearing and clamping integrated mechanical claw of the device described in this patent;
[0041] Fig. 9 It is an exploded schematic diagram of the shearing and clamping integrated mechanical claw of the device described in this patent;
[0042] Fig.10 It is a schematic diagram of the structure of the mechanical claw shearing finger of the device described in this patent;
[0043] Fig.11 It is a schematic diagram of the structure of the mechanical claw support finger of the device described in this patent;
[0044] Fig.12 is a schematic diagram of the guide area and shearing area of the device described in this patent;
[0045] Fig.13 It is a schematic diagram of the structure of the collecting device of the device described in this patent;
[0046] In the figure: 1. Navigation magnetic strip, 2. Lifting crawler chassis, 3. Control box, 4. Coordinate robot arm, 5. Shearing and clamping mechanical claw, 6. AGV magnetic navigation sensor, 7. Collection device, 81. Global depth camera, 82. Robotic arm depth camera, 21. Track wheel set, 22. Track chassis body, 23. Electric push rod, 24. Lifting support rod, 25. Installation platform, 41. Y-axis screw module, 42. Y-axis slide, 43. Z-axis electric cylinder module, 44. Z-axis mounting plate, 45. X-axis screw module, 46. Mechanical claw mounting rod, 47. Buckle, 48. X-axis slide, 49. Servo motor, 410. Servo motor with brake, 51. Shear finger, 52. Support finger, 53. Steering plate, 54. Connecting rod, 55. Mechanical claw connector, 56. Mechanical claw support Frame, 57. Servo, 58. Claw guide rail, 5101. Spring S, 5102. Clamping block S, 5103. Rivet S, 5104. Claw matrix fiber optic sensor transmitting end, 5105. Shear guide strip, 5106. Shear finger bracket, 5107. Connecting rod S, 5108. Slider S, 5109. Pad S, 5110. Blade, 5201. Spring B, 5202. Clamping block B, 5203. Rivet B, 5204. Claw matrix fiber optic sensor receiving end, 5205. Support guide strip, 5206. Support finger bracket, 5207. Connecting rod B, 5208. Slider B, 5209. Pad S, 71. Collection box, 72. Frame matrix fiber optic sensor transmitting end, 73. Frame matrix fiber optic sensor receiving end, 74. Sensor bracket. DETAILED DESCRIPTION
[0047] The present invention will be described in detail below in conjunction with the accompanying drawings.
[0048] The present invention and its implementation methods are described below, which are not restrictive and the actual implementation methods are not limited thereto. In short, if ordinary technicians in the field are inspired by it and design structural methods and embodiments similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the protection scope of the present invention.
[0049] See also Figure 1-13 This embodiment discloses a multi-coordinated multi-arm autonomous walking grape picking robot and a control method thereof, including:
[0050] See also Figure 1-5 , the device includes a navigation magnetic strip 1, a lifting crawler chassis 2, a control cabinet 3, a coordinate robot arm 4, a shearing and clamping mechanical claw 5, an AGV magnetic navigation sensor 6, a collection device 7, a global depth camera 81 and a robot arm depth camera 82;
[0051] The navigation magnetic strip 1 is laid on the ground according to the designed picking track, and the AGV magnetic navigation sensor 6 is fixedly installed on the right side of the crawler chassis body 22. The navigation magnetic strip 1 cooperates with the AGV magnetic navigation sensor 6 to realize the route navigation of the machine;
[0052] The global depth camera 81 is fixedly mounted on the rightmost side of the upper plane of the lifting crawler chassis 2 and is used to identify grapes and assign picking tasks;
[0053] The coordinate robot arm 4 is fixedly installed above the lifting crawler chassis 2, and the end of the coordinate robot arm 4 is fixedly installed with a shearing clamping mechanical claw 5 and a robot arm depth camera 82, and the robot arm depth camera 82 cooperates with the global depth camera 81 to realize the coordinated positioning of grapes;
[0054] The collecting device 7 is placed under the shearing and clamping mechanical claw 5, and is used to collect grapes and guide the coordinate mechanical arm 4 to place the grapes.
[0055] See also Figure 6 The lifting chassis 2 is an electric crawler chassis, which mainly includes two sets of crawler groups 21, a crawler chassis body 22, an electric push rod 23, a lifting support rod 24 and a mounting platform 25. Two sets of track wheel groups 23 are installed on both sides of the crawler chassis body 22. The two ends of the lifting support rod 24 are respectively installed on the crawler chassis body 22 and the mounting platform 25 in the form of hinge connection and slider connection. The two ends of the electric push rod 23 are respectively installed on the crawler chassis body 22 and the sliding end of the lifting support rod 23. The extension and retraction of the electric push rod 23 can realize the lowering and raising of the mounting platform 25. The lifting function can enable the machine to adapt to different planting heights.
[0056] See also Figure 7 The coordinate robot arm 4 has a total of 3 groups, which are arranged horizontally. The Y-axis screw module 41 of the coordinate robot arm 4 is fixed to the mounting platform 25 by a bolt group, the Z-axis electric cylinder module 43 is fixedly mounted on the Y-axis slide 42 by screws, the X-axis screw module 45 is fixedly mounted on the Z-axis mounting plate 44 by screws, the mechanical claw mounting rod 46 is fixedly mounted on the X-axis slide 42 by a buckle 47 and screws, the shearing and clamping mechanical claw 5 is installed on the end of the mechanical claw mounting rod 46 by screws, and the robot arm depth camera 82 is fixedly mounted on the mounting platform 25 by screws, and is located in front of the collection device 7.
[0057] See also Figure 8-12The shearing and clamping mechanical claw 5 is fixedly installed with a steering gear 57 and a claw guide rail 58 using a mechanical claw bracket 56 as a carrier. The steering plate 53 is fixedly connected to the output shaft of the steering gear 57. Two connecting rods 54 are connected to the steering plate 53 through a hinge. The two connecting rods are respectively connected to the shearing finger 51 and the supporting finger 52 through a hinge. At the same time, the shearing finger 51 is slidably installed on the claw guide rail 58 through a slider S5108, and the supporting finger 52 is slidably installed on the claw guide rail 58 through a slider B5208; the rotational motion of the steering gear 57 is transmitted through the steering plate 53 and the connecting rod 54, and is converted into the opposite linear motion of the shearing finger 51 and the supporting finger 52 along the claw guide rail 58, thereby realizing the shearing and clamping actions;
[0058] The shear finger 51 is supported by a shear finger bracket 5106, and its rear end is fixedly connected to a connecting rod S5107 and a shear finger slider 5108 by screws, and a shear guide strip 5105, a pad 5109 and a claw matrix optical fiber sensor transmitting end 5104 are fixedly installed on the upper plane in sequence, and a blade 5110 is fixedly installed in a blade installation groove on its side, and a clamping block S5102 is assembled in a corresponding square hole on the shear finger bracket 5106 in the form of a slider, and a rivet S5103 is fixedly installed on the clamping block S5102, and a spring S5101 is sleeved on the rivet S5103, and both ends of the spring S5101 are in contact with the clamping block S5102 and the shear finger bracket 5106 respectively;
[0059] The supporting finger 52 is supported by a supporting finger bracket 5206, and its rear end is fixedly connected to a connecting rod B5207 and a supporting finger slider 5208 by screws, and a pad 5209, a supporting guide bar 5205 and a claw matrix optical fiber sensor receiving end 5204 are fixedly installed on the upper plane in sequence, and a clamping block B5202 is assembled in the corresponding square hole on the supporting finger bracket 5206 in the form of a slider. A rivet B5203 is also fixedly installed on the clamping block B5202, and a spring B5201 is sleeved on the rivet B5203, and both ends of the spring B5201 are in contact with the clamping block B5202 and the shear finger bracket 5206 respectively.
[0060] See also Figure 8-12 The shearing guide strip 5105 and the supporting guide strip 5205 form a trapezoidal guide area that gradually opens outwards and forms a closed rectangular shearing area inwards. The guide area guides the grape stalks into the shearing area to wait for the shearing and clamping of the shearing clamping mechanical claw 5. At the same time, the claw matrix optical fiber sensor receiving end 5204 cooperates with the claw matrix optical fiber sensor receiving end 5204 to determine whether there are stalks in the shearing area, thereby realizing the inspection of whether the coordinate robot 4 is accurately in place. Due to the existence of the guide area, the requirements for the movement accuracy of the coordinate robot 4 and the positioning accuracy of the camera on the picking point can be greatly reduced;
[0061] When the shearing and holding mechanical claw 5 is in action, the shearing finger 51 and the supporting finger 52 move toward each other along the claw guide rail 58, the clamping block S5102 and the clamping block B5202 contact the fruit stalk, the spring S5101 and the spring B5201 are gradually compressed, and a holding force is applied to the two clamping blocks. The clamping force acts on the fruit stalk, clamps the fruit stalk to complete the holding action, and at the same time, the blade 5110 gradually approaches and embeds into the knife groove of the supporting finger bracket 5206 to complete the shearing action.
[0062] See also Figure 1 ,2,3,4,5,13, the collecting box 71 of the collecting device 7 is placed in the limiting groove 26 of the mounting platform 25, below the shearing and clamping mechanical claw 5, the frame matrix optical fiber sensor transmitting end 72 and the frame matrix optical fiber sensor receiving end 73 are respectively installed on both sides of the collecting box 71 through sensor brackets 74, both of which are slightly higher than the upper frame edge of the collecting box 71. The frame matrix optical fiber sensor detects the distribution of grape bunches on the upper edge of the collecting box 71, and guides the coordinate robot 4 to place the grape bunches in the collecting box 71 to ensure that the collecting box 71 is evenly filled; at the same time, it can detect whether the collecting box 71 is full, which is used to guide whether the frame needs to be replaced.
[0063] The control method and working principle of the present invention are:
[0064] See also Figure 1-13 , the work includes the following steps:
[0065] Step 1, according to the planting pattern and layout of the orchard, combined with the turning radius and working width of the machine, plan the working route, lay the navigation magnetic strip 1 on the working route, the magnetic strip can be used many times after being laid once, and then adjust the electric push rod 23 according to the working height so that the machine reaches a suitable working height; during the operation, the lifting crawler chassis 2 moves forward along the navigation magnetic strip 1 with the X-axis travel of the robot arm as the unit length under the guidance of the AGV magnetic navigation sensor 6 and the path detection system, and enters the next unit length for operation after picking a total unit length.
[0066] Step 2, the global depth camera 81 collects the image information and depth information above it, and transmits the information to the host computer located in the control box 3 through the serial port. The host computer processes the received information based on a deep learning algorithm including but not limited to yolov8, and after loading the model parameters and weights, inputs the preprocessed image into the YOLO model; performs forward propagation, and detects targets on feature maps of different scales through multiple convolution and pooling layers; performs target detection at each scale, and predicts multiple bounding boxes and corresponding category probabilities of each grid through the convolution layer; decodes the prediction of each bounding box to obtain the center coordinates, width, height and confidence score of the bounding box, and then calibrates the position of the grape bunch, and then determines the point above the center point of the upper edge of the bounding box at a certain distance as the picking point, and determines the coordinates of the picking point relative to the global depth camera 81 in combination with the depth information;
[0067] Step 3, assigning a coordinate robot 4 to each bunch of grapes according to the distribution of the grape bunches processed in step 2, wherein the picking order of the grape bunches and the motion trajectory of each robot arm are reasonably planned to avoid the motion interference between the robots.
[0068] Step 4, according to the picking tasks assigned in step 3 and the picking order and picking trajectory determined, each set of coordinate robot arms 4 starts to pick grape bunches in turn. Each picking is divided into two stages. In the first stage, the Y-axis and Z-axis coordinates of the shearing and clamping mechanical claws 5 are controlled to be roughly aligned. In the X-axis direction, the shearing and clamping mechanical claws 5 are located a certain distance behind the grape bunches; in the second stage, the depth camera 82 performs a secondary detection on the grape bunches to determine their relative coordinates. Since the robot arm depth camera 82 is close to the grape bunches at this time, the determined coordinate error is small, which can greatly improve the positioning accuracy; after the secondary positioning is completed, the robot arm is controlled to move secondary, and the grape stalks are guided into the shearing area by the guide area, and then sheared after being determined by the claw matrix fiber optic sensor. If the claw matrix fiber optic sensor confirms failure, the robot claw 5 returns to the starting point of the second stage, and picks after another detection until it is confirmed that the picking is completed;
[0069] Step 5, after cutting and clamping the grape bunches, the placing action is performed, the matrix fiber optic sensor, the frame matrix fiber optic sensor transmitting end 72 and the matrix fiber optic sensor receiving end 73 detect the distribution of the grape bunches on the upper edge of the collection box 71, and guide the coordinate robot 4 to place the grape bunches in the collection box 71 to ensure that the collection box 71 is evenly filled; when it is detected that the grape bunches on the upper edge of the collection box 71 are full, the movement is stopped and the collection box 71 is replaced; when placing, the robot fills the collection box 71 layer by layer in a prescribed placement order, and gradually lowers the height of the grape bunches in the process. When the bottom of the grape bunch contacts the bottom surface of the collection box 71 or other grape bunches, the grape bunches change from downward movement to oblique downward movement, and gradually transition from a vertical state to a horizontal state to prevent the grape berries from falling off and the skin from being scratched due to collision.
[0070] The above implementation modes are only used to illustrate the present invention, but not to limit the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the essence and scope of the present invention. Therefore, all equivalent technical solutions also belong to the scope of the present invention. The scope of patent protection of the present invention should be limited by the claims.
[0071] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.
Claims
1. A multi-coordinated multi-arm autonomous walking grape picking robot, characterized in that: The picking robot comprises a navigation magnetic strip (1), a lifting crawler chassis (2), a control cabinet (3), a coordinate mechanical arm (4), a shearing and clamping mechanical claw (5), an AGV magnetic navigation sensor (6), a collecting device (7), a global depth camera (81) and a mechanical arm depth camera (82); The navigation magnetic strip (1) is laid on the ground according to the designed picking track; The AGV magnetic navigation sensor (6) is fixedly mounted on the right side of the crawler chassis body (22), and the navigation magnetic strip (1) cooperates with the AGV magnetic navigation sensor (6) to realize the route navigation of the machine; The global depth camera (81) is fixedly mounted on the rightmost side of the upper plane of the lifting crawler chassis (2) and is used to identify grapes and assign picking tasks; The coordinate mechanical arm (4) is fixedly mounted above the lifting crawler chassis (2), the shearing clamping mechanical claw (5) and the mechanical arm depth camera (82) are fixedly mounted at the end of the coordinate mechanical arm (4), and the mechanical arm depth camera (82) cooperates with the global depth camera (81) to realize grape collaborative positioning; The collecting device (7) is placed below the shearing and clamping mechanical claw (5) and is used to collect the grapes and guide the coordinate mechanical arm (4) to place the grapes.
2. The multi-coordinated multi-arm autonomous walking grape picking robot according to claim 1, characterized in that: The lifting crawler chassis (2) is an electric crawler chassis, comprising two sets of crawler wheel assemblies (21), a crawler chassis body (22), an electric push rod (23), a lifting support rod (24) and a mounting platform (25); Two sets of track wheel assemblies (21) are installed on both sides of the track chassis body (22); The two ends of the lifting support rod (24) are respectively mounted on the crawler chassis body (22) and the mounting platform (25) in a hinge connection and a slider connection manner; The two ends of the electric push rod (23) are respectively mounted on the crawler chassis body (22) and the sliding end of the lifting support rod (24). The extension and retraction of the electric push rod (23) enables the lowering and raising of the mounting platform (25) for adjusting the height of the machine.
3. The multi-coordinated multi-arm autonomous walking grape picking robot according to claim 2, characterized in that: The coordinate robot arm (4) has a total of three groups, which are arranged horizontally. The Y-axis screw module (41) of the coordinate robot arm (4) is fixed to the mounting platform (25) by a bolt group, the Z-axis electric cylinder module (43) is fixedly mounted on the Y-axis slide (42) by screws, the X-axis screw module (45) is fixedly mounted on the Z-axis mounting plate (44) by screws, the mechanical claw mounting rod (46) is fixedly mounted on the Y-axis slide (42) by a buckle (47) and screws, the shear clamping mechanical claw (5) is fixedly mounted on the end of the mechanical claw mounting rod (46) by screws, and the robot arm depth camera (82) is fixedly mounted on the mounting platform (25) by screws and is located in front of the collection device (7).
4. The multi-eye coordinated multi-arm autonomous walking grape picking robot according to claim 3, characterized in that: The shearing and clamping mechanical claw (5) is fixedly installed with a steering gear (57) and a claw guide rail (58) using a mechanical claw bracket (56) as a carrier. The steering disc (53) is fixedly connected to the output shaft of the steering gear (57). The steering disc (53) is connected to two connecting rods (54) through a hinge. The two connecting rods (54) are respectively connected to a shearing finger (51) and a supporting finger (52) through a hinge. The shearing finger (51) is slidably installed on the claw guide rail (58) through a slider, and the supporting finger (52) is slidably installed on the claw guide rail (58) through a slider. The rotational motion of the steering gear (57) is transmitted by the steering disc (53) and the connecting rod (54) and converted into the opposite linear motion of the shearing finger (51) and the supporting finger (52) along the claw guide rail (58), thereby realizing shearing and clamping actions. The shear finger (51) uses the shear finger bracket (5106) as a bracket, and the rear end of the shear finger (51) is fixedly connected to the connecting rod S (5107) and the shear finger slider (5108) by screws. The shear guide strip (5105), the pad (5109) and the claw matrix optical fiber sensor transmitting end (5104) are fixedly installed on the plane of the shear finger (51) in sequence. The blade (5110) is fixedly installed in the blade installation groove on the side of the shear finger (51). The clamping block S (5102) is assembled in the corresponding square hole on the shear finger bracket (5106) in the form of a slider. The clamping block S (5102) is fixedly installed with a rivet S (5103), and a spring S (5101) is sleeved on the rivet S (5103). The two ends of the spring S (5101) are respectively in contact with the clamping block S (5102) and the shear finger bracket (5106); The support finger (52) uses the support finger bracket (5206) as a bracket, and the rear end of the support finger (52) is fixedly connected to the connecting rod B (5207) and the support finger slider (5208) by screws. The pad (5209), the support guide bar (5205) and the claw matrix optical fiber sensor receiving end (5204) are fixedly installed on the support finger (52) in sequence. The clamping block B (5202) is assembled in the corresponding square hole on the support finger bracket (5206) in the form of a slider. The clamping block B (5202) is also fixedly installed with a rivet B (5203), and a spring B (5201) is sleeved on the rivet B (5203). The two ends of the spring B (5201) are respectively in contact with the clamping block B (5202) and the support finger bracket (5206).
5. The multi-coordinated multi-arm autonomous walking grape picking robot according to claim 4, characterized in that: The shearing guide strip (5105) and the supporting guide strip (5205) form a trapezoidal guide area that gradually opens outwards and forms a closed rectangular shearing area inwards, and the guide area guides the grape stems into the shearing area to wait for shearing and clamping by the shearing clamping mechanical claw (5); the claw matrix optical fiber sensor receiving end (5204) cooperates with the claw matrix optical fiber sensor receiving end (5204) to determine whether there are fruit stems in the shearing area, so as to realize the inspection of whether the coordinate mechanical arm (4) is accurately in place; When the shearing and clamping mechanical claw (5) is in motion, the shearing finger (51) and the supporting finger (52) move toward each other along the claw guide rail (58), the clamping block S (5102) and the clamping block B (5202) contact the fruit stalk, the spring S (5101) and the spring B (5201) are gradually compressed, and a clamping force is applied to the two clamping blocks. The clamping force acts on the fruit stalk, clamping the fruit stalk to complete the clamping action, and at the same time, the blade (5110) gradually approaches and is embedded in the knife groove of the supporting finger bracket (5206) to complete the shearing action.
6. The multi-coordinated multi-arm autonomous walking grape picking robot according to claim 5, characterized in that: The collection box (71) of the collection device (7) is placed inside the limiting groove (26) of the installation platform (25) and is located below the shearing and clamping mechanical claw (5). The frame matrix optical fiber sensor transmitting end (72) and the frame matrix optical fiber sensor receiving end (73) are respectively installed on both sides of the collection box (71) through sensor brackets (74). The frame matrix optical fiber sensor transmitting end (72) and the frame matrix optical fiber sensor receiving end (73) are higher than the upper frame edge of the collection box (71). The frame matrix optical fiber sensor detects the distribution of grape bunches on the upper edge of the collection box (71) and is used to guide the coordinate mechanical arm (4) to place the grape bunches in the collection box (71) to ensure that the collection box (71) is evenly filled. The frame matrix optical fiber sensor detects whether the collection box (71) is full, and is used to indicate whether a frame replacement is required.
7. A control method for a multi-coordinated multi-arm autonomous walking grape picking robot, characterized in that: The method comprises the following steps: Step 1: According to the planting pattern and layout of the orchard, the operation route is planned in combination with the turning radius and working width of the machine, and the navigation magnetic strip (1) is laid on the operation route. The navigation magnetic strip (1) can be used multiple times after being laid once, and then the electric push rod (23) is adjusted according to the operation height to make the machine reach a suitable operation height; during the operation, the lifting crawler chassis (2) moves forward along the navigation magnetic strip (1) with the X-axis travel of the robot arm as the unit length under the guidance of the AGV magnetic navigation sensor (6) and the path detection system, and after picking a total unit length, it enters the next unit length for operation; Step 2: The global depth camera (81) collects image information and depth information from above, and transmits the information to a host computer located in the control box (3) through a serial port. The host computer processes the received information through a deep learning algorithm, loads model parameters and weights, and then inputs the preprocessed image into the YOLO model; performs forward propagation, and detects targets on feature maps of different scales through multiple convolution and pooling layers; performs target detection at each scale, and predicts multiple bounding boxes and corresponding category probabilities of each grid through the convolution layer; decodes the prediction of each bounding box to obtain the center coordinates, width, height and confidence score of the bounding box, and then calibrates the position of the grape bunch, and then determines the picking point above the center point of the upper edge of the bounding box, and determines the coordinates of the picking point relative to the global depth camera (81) in combination with the depth information; Step 3: According to the distribution of the grape bunches processed in step 2, a coordinate robot arm (4) is assigned to each grape bunch. By rationally planning the order of picking the grape bunches and the motion trajectory of each robot arm, the motion interference between the robots is avoided. Step 4: According to the picking tasks assigned in step 3 and the picking order and picking trajectory determined, each set of coordinate robot arms (4) starts picking grape bunches in turn. Each picking is divided into two stages. In the first stage, the Y-axis and Z-axis coordinates of the shearing and clamping mechanical claws (5) are controlled to be aligned. In the X-axis direction, the shearing and clamping mechanical claws (5) are located behind the grape bunches. In the second stage, the robot arm depth camera (82) performs a second inspection on the grape bunches to determine their relative coordinates. After the second positioning is completed, the robot arm starts to control the second movement. The grape stalks are guided by the guide area into the cutting area, and then cut after being confirmed by the claw matrix fiber optic sensor. If the claw matrix fiber optic sensor confirms failure, the robot claw (5) returns to the starting point of the second stage, and picks the grapes after another inspection until it is confirmed that the picking is completed. Step 5: After cutting and clamping the grape bunches, the placing action is performed. The matrix optical fiber sensor, the frame matrix optical fiber sensor transmitting end (72) and the matrix optical fiber sensor receiving end (73) detect the distribution of the grape bunches on the upper edge of the collection box (71), and guide the coordinate robot arm (4) to place the grape bunches in the position of the collection box (71), so as to ensure that the collection box (71) is evenly filled; when it is detected that the grape bunches on the upper edge of the collection box (71) are full, the movement is stopped and the collection box (71) is replaced; when placing, the robot arm fills the collection box (71) layer by layer in a neat manner according to the prescribed placing order, and gradually lowers the height of the grape bunches during the process. When the bottom of the grape bunch contacts the bottom surface of the collection box (71) or the remaining grape bunches, the grape bunches are changed from downward movement to oblique downward movement, and gradually transition from a vertical state to a horizontal state, so as to prevent the grape berries from falling off and the skin from being scratched due to collision.
Citation Information
Patent Citations
Automatic-navigation crawler-type mobile fruit picking robot and fruit picking method
CN102165880A
Fruit picking robot
CN112715163A
Method and device for synchronously identifying and positioning grapes and picking points thereof based on deep learning
CN114494441A
Intelligent watermelon picking robot
CN115529938A
Pick end effector of robot
CN206633036U
Cited By
Full-automatic harvesting device and method for upper tobacco leaves of flue-cured tobaccos with stems
CN119563452A
Full-automatic upper tobacco leaf strip harvesting device and method for flue-cured tobacco
CN119563452B
Self-propelled multi-arm multi-tooth flower and fruit thinning robot for table grapes
CN120615529A
Humanoid picking module and grape picking robot
CN122936094A
