Ridge-crossing four-wheel-drive-wheel type double-arm cooperative strawberry picking robot and picking method

Abstract

The invention discloses a ridge-crossing four-wheel-drive-wheel type double-arm cooperative strawberry picking robot and a picking method. The ridge-crossing four-wheel-drive-wheel type double-arm cooperative strawberry picking robot comprises a four-wheel-drive-wheel type walking platform, a picking and blowing device, a binocular depth camera, a control system, a power source and a storage basket. The four-wheel-drive-wheel type walking chassis is used for achieving ridge-crossing walking, and the picking and blowing device comprises a six-degree-of-freedom mechanical arm, an air pump, a pneumatic flexible three-finger clamping jaw, a mini air blower and a film type pressure sensor. The six-degree-of-freedom mechanical arm is installed in the center of the two sides of the four-wheel-drive-wheel type walking platform, the pneumatic flexible three-finger clamping jaw is inflated through the air cylinder to wrap the strawberries, and lossless picking is achieved. And the film pressure sensor is adhered to the inner wall of the soft finger to monitor the grabbing force. The picking method comprises the steps of three-dimensional mapping, task allocation, target identification, main axis estimation, air blowing, picking, fruit putting and intermittent walking. According to the invention, a double-arm cooperative control automatic picking mode is adopted, so that efficient and reliable automatic picking of ridge culture strawberries is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of automated strawberry picking, and in particular relates to a ridge-spanning four-wheel-drive double-arm coordinated strawberry picking robot and a picking method. Background Art

[0002] Strawberries are one of the most popular fruits due to their unique taste and rich nutrition, and are known as the "Queen of Fruits". Strawberries have a short ripening period and are large in size, so they usually need to be harvested quickly. Due to the delicate texture and fragile physical characteristics of strawberries, they are mainly picked by hand, which is extremely labor-intensive and inefficient. In addition, due to repetitive picking movements, manual picking activities can easily cause back strains and musculoskeletal problems to workers. The above problems have greatly hindered the expansion of the strawberry industry, so there is an urgent need to introduce automated picking equipment to promote the development of the strawberry planting industry.

[0003] In my country, strawberry plants are relatively short, and most of them are planted in a ground-based double-row planting mode with small ridge spacing. The strawberries grow densely, so large-scale picking equipment cannot be used. In the past, most automatic strawberry picking robots were single-armed, with low picking efficiency and high missed picking rates. Multi-arm collaborative picking has become the mainstream trend in fruit picking. At the same time, the strawberry growth environment is complex, and leaf occlusion often occurs. Existing strawberry harvesting devices will be disturbed, resulting in missed or damaged fruits. The development of an efficient and flexible strawberry harvesting robot suitable for ground ridge strawberry scenes is an urgent problem to be solved in the strawberry planting industry. Summary of the invention

[0004] In view of the problems and defects described in the above-mentioned prior art, on the one hand, the present invention provides a cross-ridge four-wheel drive dual-arm cooperative strawberry picking robot, the robot comprising: a four-wheel drive walking platform, a picking device, a blasting device, a binocular depth camera, a main controller, a power supply, and a storage basket; The wheelbase of the four-wheel-drive walking platform is equidistant from the ridge width, and the four wheels are independently driven; The picking device is fixed to the center of both sides of the side plate of the four-wheel drive walking platform frame, and the end extends to the picking area at the bottom of the four-wheel drive walking platform frame; The binocular depth camera is fixed to the end flange of the six-degree-of-freedom mechanical arm of the picking device through a mounting bracket; The main controller is placed at the top center of the four-wheel drive walking platform; The power supply is fixed to the rear of the top plate of the four-wheel drive walking platform; The storage basket is placed on the storage platform of the four-wheel drive mobile platform.

[0005] Furthermore, the four-wheel drive walking platform includes: a frame, an industrial computer, a battery pack, an encoder, a driving wheel assembly, a storage platform, a protective cover, and an emergency stop knob; The frame is a box-shaped structure spanning the ridge, which is welded and spliced ​​by combining steel frames; The industrial computer is fixedly mounted at the center of the frame by screws; The battery pack is divided into two groups, which are symmetrically fixed at the front and rear sides of the inner groove of the frame, and four extended limit blocks are used to limit the position of the battery pack; The encoder is installed at the end of the driving motor; The driving wheel assembly has four groups evenly distributed at the four corners of the frame; The storage platform is fixed to the lower rear side of the wheeled walking platform through aluminum strips extending in a triangular shape; The protective cover shell includes a top plate and side plates, the side plates are welded around the frame, and the top plate is connected to the top of the frame through a hinge; The storage basket is fixed on the storage platform by a storage basket limit block; The emergency stop knob is connected to the battery pack and the industrial computer, and is installed on the front right side of the side panel of the four-wheel drive wheeled walking platform.

[0006] Further, the picking device includes: 2 six-degree-of-freedom robotic arms, 2 air pumps, 2 pneumatic three-finger flexible grippers, a mini blower, a film pressure sensor and a mounting bracket; The six-degree-of-freedom robotic arm includes three rotating joints and three revolving joints connected alternately and driven by a built-in motor; The air pump is fixed to the end flange of the six-degree-of-freedom mechanical arm by screws; The pneumatic three-finger flexible gripper is connected to the air pump via screws and an air pipe; The mini blower is fixed to the side of the soft finger by screws and connected to the main controller by cables; The thin film pressure sensor is adhered to the fingertip position of the soft finger of the pneumatic three-finger flexible gripper; The mounting bracket is mounted on the end flange of the six-degree-of-freedom robotic arm by screws.

[0007] Furthermore, the binocular depth camera is fixed to the mounting bracket by fastening screws.

[0008] Furthermore, the main controller is placed in the four limit blocks in the center of the top plate of the four-wheel walking platform, and is connected to the battery pack, the binocular depth camera, the industrial computer, the six-degree-of-freedom robotic arm, the air pump and the pneumatic three-finger flexible gripper through cables; the main controller is connected to the binocular depth camera, the six-degree-of-freedom robotic arm, the air pump, the mini blower and the industrial computer of the four-wheel drive wheeled walking platform through a serial port.

[0009] Further, the driving wheel assembly includes: a steering motor, a gear transmission device, a driving belt assembly, a worm gear reducer, a supporting foot rod, a wheel reducer, a rubber wheel and a driving motor; The output shaft of the steering motor is connected to the input end of the worm gear reducer; the encoder is installed on the output shaft of the drive motor; the drive motor is installed on the driving wheel of the transmission belt assembly, connected to the driven wheel through the belt, and the support foot rod passes through the driven wheel shaft hole. The gear transmission device is meshed together through gears; The transmission belt assembly comprises: a driving wheel, a circular rubber serrated belt and a driven wheel; the circular rubber serrated belt is respectively sleeved on the driving wheel and the driven wheel; The support foot rod passes through the bearing at the bottom of the frame, the top is fixedly connected to the driven wheel shaft hole, and the bottom is fixed to the wheel reducer by bolts; The wheel reducer is fixed to the end of the supporting foot rod by bolts; the transmission shaft of the wheel reducer is inserted into the axle hole of the rubber wheel, and the position of the wheel and the transmission shaft is limited by pins. The wheel reducer contains a bevel gear assembly for converting the transmission direction.

[0010] Furthermore, the rubber wheel adopts a tire made of pneumatic rubber and a wheel hub made of aluminum alloy.

[0011] Furthermore, the pneumatic three-finger flexible clamp includes: 3 soft fingers, a finger clamp, a finger base, a bottom disc flange and an inflation valve. A groove is set between the upper and lower parts of the soft finger, and the soft finger is clamped in the finger clamp by the groove. The finger clamp is installed on the finger base through a fastener; the surface of each flexible finger is made of silicone rubber; the finger base is evenly fixed on the bottom disc flange by screws; the side wall of the finger base has a threaded hole that is fixedly connected to the end of the inflation valve.

[0012] On the other hand, the present invention further provides a picking method, using the above-mentioned cross-ridge four-wheel drive double-arm collaborative strawberry picking robot, comprising the following steps: Step S1: Start the robot, and the main controller sends an initialization command to control the robot arm, air pump, and end gripper to return to the initial position, and start the binocular depth camera to receive video stream images in real time; Step S2: two robotic arms respectively surround the bottom half of the frame to capture images of the strawberry area at the bottom of the frame and transmit the image information to the main controller to establish a three-dimensional map; Step S3: The main controller uses the trained deep learning network to identify the ripe strawberry and aligns it with the acquired depth information to complete the identification of the three-dimensional coordinates and contour of the strawberry; Step S4: the main controller distributes the strawberries in the area to the two robotic arms according to the obtained three-dimensional coordinates of the strawberries using an optimization algorithm based on the principle of the fastest harvesting speed; Step S5: driving and controlling the integrated robotic arm to pick the strawberries according to the strawberry position information and harvesting order transmitted by the main controller; the robotic arm internal drive plans the path, drives the three-finger flexible gripper to move forward to the specified position, and uses the binocular depth camera to fine-tune the position of the end of the robotic arm; Step S6: the main controller outputs a continuous voltage signal, and the mini blower works to blow away the leaves covering the surface of the strawberry, so that the fruit is fully exposed to the field of view of the binocular depth camera; Step S7: the main controller obtains the strawberry picking axis; the position of the three-finger flexible gripper is adjusted according to the strawberry picking axis, the soft fingers are inflated by the air pump to bend to wrap the fruit, one of the three fingers is fixed at the bottom of the picking axis, and the other two fingers hold the widest part of the fruit on opposite sides of the strawberry; at the same time, the main controller no longer outputs a signal, and the mini blower stops working; Step S8: The main controller sends an inflation command, and the air pump inflates the soft fingers of the three-finger flexible gripper, and the thin film pressure sensor transmits the piezoelectric signal to the main controller through the cable to monitor the gripping force of the gripper in real time; when the gripping force exceeds the pressure threshold, the six-degree-of-freedom robotic arm pulls the strawberry outward, and the three-finger flexible gripper rotates to separate the target strawberry from the fruit stem; Step S9: The main controller sends the storage basket position to the robotic arm, instructing the integrated drive and control robotic arm to drive the three-finger flexible gripper to carry the picked strawberries to the storage basket position; Step S10: the main controller sends an air suction command to the air pump, and the air pump continues to inhale until the three-finger flexible gripper returns to its original position, and the picked strawberries are placed in the storage basket; Step S11: repeating steps S5-S9, when all strawberries in the target area are picked in the distribution order of step S4, the main controller sends a stepping instruction to the industrial computer of the four-wheel drive walking platform, and the four-wheel drive walking platform obtains the walking distance in real time according to the encoder until it stops walking after walking a vehicle body length; Step S12: Repeat steps S2-S10 until the strawberry harvest is completed.

[0013] Furthermore, in step S7, the main controller obtains the strawberry picking axis, specifically including: obtaining a semantic segmentation image of the strawberry using a deep learning network; estimating the strawberry posture using a traversal method, the traversal method extracts the image of the ripe fruit mask according to the semantic segmentation result and extracts the ripe fruit boundary in pixels, calculates the length between two pixel points and finds the line with the longest distance, which is the strawberry picking axis.

[0014] The present invention has the following beneficial technical effects: (1) The present invention fixes a mini blower on the side of the soft finger, which moves along with the pneumatic flexible three-finger gripper. When the strawberry is blocked by leaves, air is continuously blown on the surface of the fruit, exposing the fruit. The ripe type of the strawberry and the outline of the ripe fruit can then be further identified through a binocular depth camera, thereby guiding the posture adjustment of the pneumatic flexible three-finger gripper.

[0015] (2) The present invention uses a dual-arm collaborative operation method combined with an all-round mapping method to improve the completeness of picking coverage and picking efficiency. The soft fingers of the present invention have a large contact area with the strawberry, so even if the fruit axis has a certain deviation relative to the tip of the strawberry, it does not affect picking. On the contrary, due to the large contact area, the posture self-adjusts, which is more reliable for strawberry picking. The traversal algorithm proposed in the present invention estimates the speed of the strawberry main axis faster, so the three-finger flexible gripper at the end combined with the strawberry posture estimation method can greatly improve the speed and reliability of strawberry picking.

[0016] (3) The present invention is linked by a six-degree-of-freedom robotic arm, a cylinder and a three-finger flexible gripper, which can pick strawberry fruits in the entire area below the frame and is equipped with an intermittent stepping walking mode to achieve a high degree of automation of the strawberry picking robot. BRIEF DESCRIPTION OF THE DRAWINGS The present invention has the following accompanying drawings: Figure 1 Side view of the cross-ridge four-wheel drive double-arm cooperative strawberry picking robot of the present invention Figure 2 Rear view of the cross-ridge four-wheel drive double-arm cooperative strawberry picking robot of the present invention Figure 3 Schematic diagram of the four-wheel drive walking platform of the present invention Figure 4 Schematic diagram of the driving and steering wheel assembly of the present invention Figure 5 Schematic diagram of the picking device of the present invention Figure 6 Schematic diagram of the pneumatic three-finger flexible gripper of the present invention Figure 7 The picking steps of the cross-ridge four-wheel drive double-arm cooperative strawberry picking robot of the present invention The components in the drawings are marked as follows: 1. Four-wheel drive walking platform; 2. Picking device; 3. Binocular depth camera; 4. Main controller; 5. Power supply; 6. Storage basket; 11. Frame; 12. Industrial computer; 13. Battery pack; 14. Encoder; 15. Drive and steering wheel assembly; 16. Storage platform; 17. Protective cover; 18. Emergency stop knob; 21. Six-degree-of-freedom robotic arm; 22. Air pump; 23 Pneumatic three-finger flexible gripper; 24. Mini blower; 25. Thin film pressure sensor; 26. Mounting bracket; 151, driving motor; 152, gear transmission device; 153, transmission belt assembly; 154, worm gear reducer; 155, support foot rod; 156, wheel reducer; 157, rubber wheel; 1531, driving wheel; 1532, round rubber serrated belt; 1533, driven wheel; 161, storage basket limit block; 171, top plate; 172, side plate; 231, soft finger; 232, finger clamp; 233, finger base; 234, bottom disc flange; 235, inflation valve. 1711, controller limit block; 1712, power box limit block. DETAILED DESCRIPTION

[0018] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0019] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0020] Specific implementation method: In order to make the researchers and workers in this technical field better understand the present invention The present invention is further described below in conjunction with specific embodiments and accompanying drawings: As attached Figures 1 to 4As shown, a wheeled dual-arm system strawberry picking robot includes a four-wheel drive wheeled walking platform 1, a picking device 2, a binocular depth camera 3, a main controller 4, a power supply 5 and a storage basket 6. The wheelbase of the four-wheel drive wheeled walking platform 1 is equidistant from the ridge width, so it can walk across the ridges. The four wheels are independently driven, with sufficient driving power and convenient steering. The picking device 2 is fixed to the center of both sides of the side plate 172 of the four-wheel drive walking platform frame 11 by bolts, and the end extends to the picking area at the bottom of the frame 11, which is used for picking strawberries and blocking the leaves from blowing up. The binocular depth camera 3 is fixed to the end flange of the driving and controlling integrated six-degree-of-freedom mechanical arm 21 of the picking device through a mounting bracket 26, which is used for three-dimensional mapping and guiding the mechanical arm to pick. ; The mini blower 24 is fixedly mounted on the side of the soft finger 231 of the flexible three-finger gripper 23 of the picking device 2, and is used to blow away the blocking leaves; the main controller 4 is placed in the center of the top of the four-wheel walking platform; the power supply 5 is fixed to the tail of the top plate 11 of the four-wheeled walking platform 1, and is placed symmetrically. Four extended power limit blocks 1712 are used to limit its position to avoid slippage during the movement of the platform; the storage basket 6 is placed in the storage basket limit block 161 of the storage platform 16, and a certain thickness of The foam pad prevents the strawberries from falling and getting damaged, ensuring the quality of the harvested strawberries.

[0021] As attached Figures 1 to 3As shown, the four-wheel drive walking platform 1 includes a frame 11, an industrial computer 12, a battery pack 13, an encoder 14, a driving and steering wheel assembly 15, a storage platform 16, a protective cover shell 17, and an emergency stop knob 18. The frame 11 is a box-shaped structure that spans the ridges and is assembled by welding a steel frame. A picking area that can pass through the strawberry ridges is formed below it, which is used to support the picking and blowing device 2, the storage platform 16, and the main controller 4. The industrial computer 12 is fixedly installed inside the frame 11 by screws, and is separated from the overall frame by studs to avoid electromagnetic induction between the industrial computer and the internal circuit of the vehicle, and is used to control the walking and steering of the vehicle; the battery pack 13 is divided into two groups and symmetrically fixed on the front and rear sides of the internal groove of the frame 11, respectively, to provide power for the driving wheel assembly 15, the picking device 2, the industrial computer 12, and the main controller 4; the encoder 14 is installed at the end of the drive motor, and is used to convert the linear displacement into an electrical signal, and then convert the electrical signal into a counting pulse, and the number of counting pulses is used to represent the displacement. , the number of count pulses is transmitted to the industrial computer 11, so that the intermittent stepping motion of the four-wheel drive vehicle 1 can be controlled; the driving and steering wheel assembly 15 has four groups evenly distributed at the four corners of the frame, which are used to drive the four-wheel drive walking platform 1 to move, and the four steering reducers can be independently controlled. Each wheel rotates and deflects autonomously according to a preset program; the storage platform 16 is fixed to the lower rear side of the tail of the wheeled walking platform through an aluminum strip in a triangular extension. The installation height is lower than the bottom height of the frame 11 and higher than the strawberry ridge. It is used to place the storage basket and avoid contact with the ridge during walking. A collision occurs; four storage basket limit blocks 161 are installed at the four corners of the storage platform 16 to limit the position of the storage basket 6 and prevent the storage basket 6 from sliding during walking; the protective cover shell 17 includes a top plate 171 and side plates 172, the side plates 172 are welded around the frame, the top plate 171 is connected to the top of the frame 11 through a hinge, and the protective cover shell 17 is used to isolate the internal circuit and the industrial computer from the external environment. At the same time, the robot arm 21 is fixed to the center of both sides of the side plates by bolts; four limit blocks 1712 are installed above the top plate 11 according to the size of the main controller , used to place the main controller; there are two storage baskets 6, which are placed on the storage platform 16 through the storage basket limit block 161 to collect the picked ripe strawberries; the main controller 4 is placed above the top plate 171 through the limit block 1711, which is used for the linkage of the binocular depth camera, the picking system and the four-wheeled platform; the emergency stop knob 18 is connected to the battery pack 13 and the industrial computer 12, and is installed on the right front of the side panel 172 of the four-wheeled mobile platform 1. Its function is to cut off the power supply of the equipment in an emergency situation, so as to avoid mechanical accidents or casualties. When restarting the device, just rotate it 45° clockwise and release it, the button pops up, and the power supply of the device is restored.

[0022] As attached Figure 5As shown, the picking device 2 includes two integrated six-degree-of-freedom mechanical arms 21, two air pumps 22, two pneumatic three-finger flexible grippers 23, a mini blower 24 and a film pressure sensor 25. The six-degree-of-freedom integrated mechanical arm 21 includes three rotating joints and three swivel joints connected alternately. The rotating joints and swivel joints are driven by built-in motors. The range of motion covers the semi-enclosed area on the side where the frame is installed, which is used to control the picking and placing movement of the mechanical arm at the bottom of the frame. The six-degree-of-freedom integrated mechanical arm 21 is fixed to the center of both sides of the bottom of the frame 11 by bolts, and is used to drive the pneumatic flexible three-finger gripper to pick, move and place strawberries; the air pump 22 is fixed to the end flange of the mechanical arm by screws, and is used to inflate and inhale the pneumatic three-finger flexible gripper 23; the pneumatic three-finger flexible gripper 23 is connected to the air pump 22 by screws and air pipes, and the strawberries are grasped and placed by inflating and inhaling the air pump. The mini blower 24 is fixed to the side of the soft finger 231 by screws, and is connected to the main control 4 by cables, and is used to blow away the leaves on the upper surface of the strawberry that are blocked, so that the strawberry fruit is completely exposed.

[0023] As attached Figure 1 , 2 As shown in Figures 3 and 5, the binocular depth camera 3 serves as a visual recognition terminal, and has a threaded hole at the tail end thereof, which is fixed to a mounting bracket 26 by fastening screws, and is used for sensing, identifying and locating strawberries and for hand-eye calibration with a six-degree-of-freedom drive-integrated robotic arm 21. The power of the binocular depth camera 3 is provided by a main controller 4 through a USB cable, and the data transmission function is realized by the USB cable; the mounting bracket 26 is fixed to the end flange of the six-degree-of-freedom drive-integrated robotic arm 21 by screws, and the mounting bracket 26 is a freely rotatable platform, which is used to adjust the pitch and yaw angles of the binocular depth camera 3, and the adjusted angle enables the field of view of the binocular depth camera to include the tail end of the pneumatic flexible three-finger gripper and most of its front range.

[0024] As attached Figure 1 , 2As shown, the main controller 4 is placed in the four limit blocks 1711 in the center of the top plate 171 of the four-wheel walking platform, and is connected to the lower battery pack 13, the binocular depth camera 3, the industrial computer 11, the six-degree-of-freedom mechanical arm 21, the air pump 22 and the pneumatic flexible three-finger gripper 23 through cables. The main controller 4 is equipped with a data acquisition card for collecting information from the industrial computer 12, the binocular depth camera 3 and the thin film pressure sensor 25 to achieve fast integrated control of identification and picking; the main controller 4 is connected to the six-degree-of-freedom mechanical arm 21 through a serial port to guide the movement of the mechanical arm in the working area; the main controller 4 is connected to the mini blower 24 through a serial port to supply power to it while controlling the working state of the mini blower 24 to achieve the operation of blowing up the blades, thereby further identifying and picking the blocked strawberries; the main controller 4 is connected to the picking device 2 through a serial port to control the movement and posture adjustment of the picking device 2; The main controller 4 is connected to the industrial computer 12 of the four-wheeled walking platform 1 through a serial port, and is used to transmit a stepping signal to the industrial computer 12 of the four-wheeled walking platform to control the platform to move forward a vehicle body distance for picking after completing the picking task in the target area. Figure 4As shown, the driving wheel assembly includes a steering motor 151, a gear transmission device 152, a transmission belt assembly 153, a worm gear reducer 154, a supporting foot rod 155, a wheel reducer 156, a rubber wheel 157 and a driving motor 158. The output shaft of the steering motor 151 is connected to the input end of the worm gear reducer 153 for torque transmission; the worm gear reducer 154 uses a lead screw and a gear to obtain a large torque, thereby driving the vehicle body to move; the encoder 14 is installed on the output shaft of the driving motor 151, and is used to measure the rotation distance of the rubber wheel 156; the gear transmission device 152 is meshed with gears, and is used to support the foot rod 155 to rotate around a vertical axis; the driving pulley assembly 153 includes a driving wheel 1531, a circular rubber serrated belt 1532 and a driven wheel 1533. The rubber serrated belt 1532 is respectively mounted on the driving wheel 1531 and the driven wheel 1533, and is used to amplify the rotation multiple and transmit the torque to the wheel reducer 156; the support foot rod 155 passes through the bearing at the bottom of the frame 11, and the top is fixed to the axial hole of the driven wheel 1533, and the bottom is fixed to the wheel reducer 156 with bolts, and is used to drive the walking platform to move; the support foot rod 155 can be replaced to adapt to the planting of strawberries with different ridge heights, and the bearing is used to reduce the friction between the support foot rod and the frame; the wheel reducer 156 and the support foot rod are connected to the frame. The end of the rod 155 is fixed by bolts, and contains a bevel gear assembly inside for converting torque and driving the wheel to roll; the transmission shaft of the wheel reducer 156 is inserted into the axial hole of the wheel 157, and the transmission shaft is a hexagonal solid column. A pin hole is set at the connection between the wheel and the transmission shaft. The position of the wheel and the transmission shaft is limited by a pin to prevent axial displacement of the wheel. The transmission shaft pin is coaxial with the wheel bearing, and the inner ring of the bearing rotates with the transmission shaft to control the robot to walk; the rubber wheel 157 uses an inflatable rubber tire and an aluminum alloy wheel hub.

[0025] like Figure 6 As shown, the pneumatic three-finger flexible gripper 23 includes three soft fingers 231, a finger sleeve 232, a finger base 233, a bottom disc flange 234 and an inflation valve 235. A groove is set between the upper and lower parts of the soft finger 231. The upper part of the finger is a hollow structure for inflation and inhalation, and the lower part is a solid rubber, which can be nested in the finger sleeve; the finger sleeve 232 has an inward concave setting, which can clamp the solid rubber at the lower part of the soft finger inside to prevent the soft finger from shaking; the finger sleeve 232 is installed on the finger base 233 through a fastener, and the finger epidermis is made of silicone rubber to prevent crushing or scratching when grabbing strawberries; the finger base 233 is evenly fixed on the bottom disc flange 234 by screws, and a threaded hole is opened on the side wall of the finger base to be fixedly connected to the end of the inflation valve 235; the air pump 22 divides three air pipes from the outlet and connects to the three inflation valves to achieve inflation and inhalation of the fingers.

[0026] The thin film pressure sensor 25 is adhered to the fingertip position of the soft finger 231 of the pneumatic three-finger flexible clamp 23, and transmits the piezoelectric signal to the main controller 4 in real time, which is used to monitor the gripping force of the strawberry in real time. When the pressure exceeds the set threshold, the inflation of the soft finger 231 is stopped. The use of the thin film pressure sensor avoids the problems of crushing the strawberries due to excessive picking force or grasping failure due to insufficient picking force during the picking process.

[0027] Based on the above-mentioned cross-ridge four-wheel drive double-arm collaborative strawberry picking robot, the present invention also provides a method of use, comprising the following steps: Step S1: Start the robot and the binocular camera, and initialize the states of the robot arm 21, the air pump 22, and the end gripper 23.

[0028] Step S2: The two robot arms 21 respectively surround the bottom half of the frame 11 in a semi-enclosed manner to take pictures of the terrain in the target area at equal intervals, thereby establishing a three-dimensional terrain map.

[0029] Step S3: The main controller 4 uses the trained deep learning network to identify the strawberry category and plane position, and aligns it with the acquired depth information to mark the strawberry position.

[0030] Step S4: the main controller 4 distributes the strawberries in the area to the two robotic arms 21 according to the obtained three-dimensional coordinates of the strawberries using an optimization algorithm with the principle of the fastest harvesting speed.

[0031] Step S5: The main controller 4 sends the strawberry harvesting order and the corresponding posture information to the integrated driving and controlling robot arm 21. The robot arm 21 internally drives and plans the global and local picking paths, drives the three-finger flexible gripper 23 to move forward to the specified position, and uses the binocular depth camera 3 to fine-tune the position of the robot arm end 21.

[0032] Step S6: Determine whether the target strawberry is blocked. If not, directly execute step S7. If blocked, the main controller 4 outputs a continuous voltage signal to the mini blower 26. The blower continues to work to blow away the leaves blocking the surface of the strawberry, so that the blocked fruit is fully exposed to the field of view of the binocular depth camera 3, so as to facilitate subsequent identification and picking.

[0033] Step S7: Obtain the semantic segmentation image of strawberries according to the deep learning network, extract the boundary pixels of ripe strawberries, calculate the length between all pixels using the pixel traversal method, find the line with the longest distance, and set the line segment as the strawberry picking axis. One of the fingers of the three-finger flexible clamp 23 is against the bottom of the strawberry picking axis, and the other two fingers are against the opposite sides of the strawberry. At the same time, the main controller no longer outputs a signal, and the mini blower 24 stops working.

[0034] Step S8: The main controller 4 sends an inflation command to the three-finger flexible gripper 23, so that the air pump 22 inflates the soft fingers 231 of the three-finger flexible gripper 23, and the thin film piezoelectric sensor 25 transmits the piezoelectric signal to the main controller 4 through the cable to monitor the gripping force of the gripper in real time. When the gripping force reaches the pressure threshold range, the main controller stops sending instructions to the flexible gripper, and the strawberry is completely wrapped to achieve the capture of the strawberry. After that, the six-degree-of-freedom robot arm 21 pulls the strawberry outward, and the three-finger flexible gripper 23 rotates to separate the target strawberry from the fruit stem.

[0035] Step S9: the robot arm 21 drives the three-finger flexible gripper 23 to carry the picked strawberries into the storage basket 6 according to the position of the storage basket 6 sent by the main controller 4 .

[0036] Step S10: the main controller 4 sends an air suction command to the air pump 22 , and the air pump 22 continues to inhale until the three-finger flexible clamp 23 returns to its original position. At this time, the picked strawberries are placed in the storage basket 6 .

[0037] Step S11: Repeat steps S5-S9 until all strawberries in the target area are picked in the order assigned in step S4. When the strawberries in the target area are picked, the main controller 4 sends a stepping instruction to the industrial computer 11 of the four-wheeled walking platform 1, and the four-wheeled walking platform 1 uses the encoder 14 to monitor the walking distance until it stops after walking a vehicle length, thereby realizing intermittent stepping picking.

[0038] Step S12: Repeat steps S2-S10 until the strawberry harvest is completed.

[0039] As mentioned above, the picking process is shown in the figure. When performing the picking task, first start the power supply, so that the six-degree-of-freedom robot arm, air pump and end gripper return to the initial position, start the binocular depth camera to receive the video stream image in real time, and then the picking devices of the robot arm on both sides of the frame face the areas on both sides of the ridge to pick at the same time. The robot arms on both sides move around the bottom of the frame in a semi-enclosed manner on the ridge surface. During the movement, the binocular depth camera installed at the end of the robot arm repeatedly shoots the target area and sends it to the main controller to construct a three-dimensional environment map of the picking area. The main controller uses the trained target recognition network to identify the strawberry maturity information based on the constructed three-dimensional environment map, and identifies the strawberry position based on the information returned by the binocular depth camera. The optimization algorithm is used to assign the strawberry serial number and order to the two robot arms with the shortest picking time as the optimization goal. After receiving the strawberry information from the main controller, the two integrated drive-control robotic arms plan the picking path according to the internal path planning algorithm, drive the onboard air pump and the flexible three-finger gripper to the target position, and at the same time, the binocular depth camera feeds back the strawberry position information to the main controller in real time to guide the end position of the robotic arm to fine-tune, so as to achieve the recognition and positioning of the target fruit in the near field by the "hand-eye" visual system. When the three-finger flexible gripper reaches the specified position, the main controller calculates the strawberry posture information based on the strawberry image obtained by the binocular depth camera, and further adjusts the gripper posture. When the gripper posture and the strawberry posture are aligned, the air pump inflates the soft finger to bend it, and at the same time, the thin film pressure sensor attached to the inner side of the finger surface transmits the piezoelectric signal back to the main controller in real time. When the pressure threshold is exceeded, the six-degree-of-freedom robotic arm is used to pull the strawberry outward, and the three-finger flexible gripper rotates to separate the target strawberry from the fruit stem. After the picking is completed, the main control sends the storage basket position to the robotic arm, and the robotic arm sends the picked strawberries to the storage basket according to the internal path planning algorithm. After reaching the storage basket position, the main controller sends a zeroing command to the air pump, and the air pump inhales to return the soft fingers to their original positions, thereby releasing the strawberries. The main controller then sends the location information of the next strawberry to the robotic arm. When the two robotic arms have completed the picking of strawberries in the current area in sequence, the main controller sends a stepping motion command to the industrial computer of the four-wheel drive wheeled mobile platform. The industrial computer controls the four-wheel drive wheeled walking platform to move forward a vehicle length, and uses the encoder to monitor the platform's moving distance in real time. When the platform reaches the specified position, repeat the above operation to perform the next round of picking tasks.

[0040] In the description of the present invention, the terms "install", "connect", "fix" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a direct connection, an indirect connection through an intermediate medium, or the internal connection of two elements. The directions or positional relationships indicated by the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inside", "outside" and the like are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A ridge-crossing four-wheel drive dual-arm collaborative strawberry picking robot, characterized in that: The robot comprises: a four-wheel drive walking platform (1), a picking device (2), a binocular depth camera (3), a main controller (4), a power supply (5), and a storage basket (6); The wheelbase of the four-wheel-drive walking platform (1) is equidistant from the ridge width, and the four wheels are independently driven; The picking device (2) is fixed to the center of both sides of the side plate (172) of the four-wheel drive walking platform frame (11), and the end thereof extends to the picking area at the bottom of the four-wheel drive walking platform frame (11); The binocular depth camera (3) is fixed to the end flange of the six-degree-of-freedom mechanical arm (21) of the picking device via a mounting bracket (25); The main controller (4) is placed at the top center of the four-wheel drive wheeled walking platform (1); The power source (5) is fixed to the rear of the top plate of the four-wheel drive walking platform (1); The storage basket (6) is placed on a storage platform (16) of a four-wheel drive mobile platform.

2. The harvesting robot according to claim 1, characterized in that: The four-wheel drive walking platform (1) comprises: a frame (11), an industrial computer (12), a battery pack (13), an encoder (14), a driving and steering wheel assembly (15), a storage platform (16), a protective cover (17), and an emergency stop knob (18); The vehicle frame (11) is a box-shaped structure that spans the ridge and is formed by welding and splicing steel frames; The industrial computer (12) is fixedly mounted at the center of the vehicle frame (11) by means of screws; The battery pack (13) is divided into two groups, which are symmetrically fixed at the front and rear sides of the inner groove of the frame (11), and four extended limit blocks (1712) are used to limit the position of the battery pack (13); The encoder (14) is installed at the end of the drive motor; The driving and steering wheel assemblies (15) have four groups evenly distributed at the four corners of the frame (11); The storage platform (16) is fixed to the lower rear side of the tail of the wheeled walking platform by means of aluminum strips extending in a triangular shape; The protective cover shell (17) comprises a top plate (171) and side plates (172), wherein the side plates (172) are welded around the frame, and the top plate (171) is connected to the top of the frame (11) via a hinge; The storage basket (6) is fixed on the storage platform (16) via a storage basket limit block (161); The emergency stop knob (18) is connected to the battery pack (13) and the industrial computer (12), and is installed on the front right side of the side panel (172) of the four-wheel drive walking platform (1).

3. The harvesting robot according to claim 2, characterized in that: The picking device (2) comprises: two six-degree-of-freedom mechanical arms (21), two air pumps (22), two pneumatic three-finger flexible grippers (23), a mini blower (24), a film pressure sensor (25) and a mounting bracket (26); The six-degree-of-freedom mechanical arm (21) comprises three rotating joints and three swivel joints connected alternately and driven by a built-in motor; The air pump (22) is fixed to the end flange of the six-degree-of-freedom mechanical arm (21) by means of screws; The pneumatic three-finger flexible clamp (23) is connected to the air pump (22) via screws and an air pipe; The mini blower (24) is fixed to the side of the soft finger (231) by means of screws, and is connected to the main controller (4) by means of a cable; The thin film pressure sensor (25) is adhered to the fingertip position of the soft finger (231) of the pneumatic three-finger flexible gripper (23); The mounting bracket (26) is mounted on the end flange of the six-degree-of-freedom mechanical arm (21) by means of screws.

4. The harvesting robot according to claim 3, characterized in that: The binocular depth camera (3) is fixed to the mounting bracket (26) by means of fastening screws.

5. The harvesting robot according to claim 4, characterized in that: The main controller (4) is placed in four central limit blocks (1711) of the top plate (171) of the four-wheel walking platform, and is connected to the battery pack (13), the binocular depth camera (3), the industrial computer (12), the six-degree-of-freedom robotic arm (21), the air pump (22) and the pneumatic three-finger flexible gripper (23) through cables; the main controller (4) is connected to the binocular depth camera (3), the six-degree-of-freedom robotic arm (21), the air pump (22), the mini blower (24) and the industrial computer (12) of the four-wheel drive wheeled walking platform (1) through a serial port.

6. The harvesting robot according to claim 5, characterized in that: The driving and steering wheel assembly (15) comprises: a steering motor (151), a gear transmission device (152), a driving belt assembly (153), a worm gear reducer (154), a supporting foot rod (155), a wheel reducer (156), a rubber wheel (157) and a driving motor (158); The output shaft of the steering motor (151) is connected to the input end of the worm gear reducer (154); the encoder (14) is mounted on the output shaft of the drive motor (158); the drive motor (158) is mounted on the driving wheel (1531) of the transmission belt assembly, and is connected to the driven wheel (1533) via a belt (1532); the support foot rod (155) passes through the shaft hole of the driven wheel (1533); The gear transmission device (152) is meshed together through gears; The transmission belt assembly (153) comprises: a driving wheel (1531), a circular rubber serrated belt (1532) and a driven wheel (1533); the circular rubber serrated belt (1532) is respectively sleeved on the driving wheel (1531) and the driven wheel (1533); The supporting foot rod (155) passes through the bearing at the bottom of the vehicle frame (11), the top is fixedly connected to the shaft hole of the driven wheel (1533), and the bottom is fixed to the wheel reducer (156) by bolts; The wheel reducer (156) is fixed to the end of the support foot rod (155) by means of bolts; the transmission shaft of the wheel reducer (156) is inserted into the shaft hole of the rubber wheel (157), and the positions of the wheel and the transmission shaft are restricted by means of pins.

7. The harvesting robot according to claim 6, characterized in that: The rubber wheel (157) is made of a tire made of pneumatic rubber and a wheel hub made of aluminum alloy.

8. The harvesting robot according to claim 7, characterized in that: The pneumatic three-finger flexible clamp (23) comprises: three soft fingers (231), a finger clamp (232), a finger base (233), a bottom disc flange (234) and an inflation valve (235); a groove is provided between the upper and lower parts of the soft fingers (231), and the soft fingers are clamped in the finger clamp (232) by means of the groove; the finger clamp (232) is mounted on the finger base (233) by means of a fastener; the surface of each flexible finger is made of silicone rubber; the finger base (233) is evenly fixed to the bottom disc flange (234) by means of screws; and a threaded hole is provided on the side wall of the finger base (233) to be fixedly connected with the end of the inflation valve (235).

9. A picking method, using the ridge-crossing four-wheel drive dual-arm cooperative strawberry picking robot according to any one of claims 1 to 8, characterized in that: The steps include: Step S1: Start the robot, and the main controller sends an initialization command to control the robot arm (21), the air pump (22), and the end gripper (23) to return to the initial position, and start the binocular depth camera (3) to receive video stream images in real time; Step S2: two robotic arms (21) respectively surround the bottom half of the frame (11) to capture images of the strawberry area at the bottom of the frame and transmit the image information to the main controller (4) to establish a three-dimensional map; Step S3: the main controller (4) uses the trained deep learning network to identify the ripe strawberry and aligns it with the acquired depth information to complete the identification of the three-dimensional coordinates and contour of the strawberry; Step S4: the main controller (4) allocates the strawberries in the area to the two robotic arms (21) based on the obtained three-dimensional coordinates of the strawberries using an optimization algorithm and taking the principle of the fastest harvesting speed; Step S5: driving the integrated robotic arm (21) to pick the strawberries according to the strawberry position information and harvesting order transmitted by the main controller (4); the robotic arm (21) internally drives the planned path, drives the three-finger flexible gripper (23) to move forward to the designated position, and simultaneously uses the binocular depth camera (3) to fine-tune the position of the end of the robotic arm (21); Step S6: the main controller (4) outputs a continuous voltage signal, and the mini blower (24) works to blow away the leaves that block the surface of the strawberry, so that the fruit is fully exposed within the field of view of the binocular depth camera (3); Step S7: the main controller (4) obtains a strawberry picking axis; the position of the three-finger flexible gripper (23) is adjusted according to the strawberry picking axis, the soft fingers (231) are inflated by the air pump to bend and wrap the fruit, one of the three fingers is fixed at the bottom of the picking axis, and the other two fingers hold the widest part of the fruit on opposite sides of the strawberry; at the same time, the main controller no longer outputs a signal, and the mini blower (24) stops working; Step S8: the main controller sends an inflation command, the air pump (22) inflates the soft fingers (231) of the three-finger flexible gripper (23), and the thin film pressure sensor (25) transmits the piezoelectric signal to the main controller (4) through the cable to monitor the gripping force of the gripper in real time; when the gripping force exceeds the pressure threshold, the six-degree-of-freedom robot arm (21) pulls the strawberry outward, and the three-finger flexible gripper (23) rotates to separate the target strawberry from the fruit stem; Step S9: the main controller (4) sends the position of the storage basket (6) to the robotic arm (21), instructing the integrated driving and controlling robotic arm (21) to drive the three-finger flexible gripper (23) to carry the picked strawberries to the position of the storage basket (6); Step S10: the main controller (4) sends an air suction command to the air pump (22), and the air pump (22) continues to inhale until the three-finger flexible clamp (23) returns to its original position, and the picked strawberries are placed in the storage basket (6); Step S11: Steps S5-S9 are repeated. When all strawberries in the target area are picked in the order allocated in step S4, the main controller (4) sends a stepping instruction to the industrial computer (11) of the four-wheel drive walking platform (1). The four-wheel drive walking platform (1) obtains the walking distance in real time according to the encoder (14) until it stops walking after traveling a vehicle body length. Step S12: Repeat steps S2-S10 until the strawberry harvest is completed.

10. A picking method according to claim 9, characterized in that: In step S7, the main controller (4) obtains the strawberry picking axis, specifically comprising: obtaining a semantic segmentation image of the strawberry using a deep learning network; estimating the posture of the strawberry using a traversal method, extracting an image of a ripe fruit mask based on the semantic segmentation result and extracting a ripe fruit boundary in pixels, calculating the length between two pixel points and finding the line with the longest distance therefrom, which is the strawberry picking axis.

Images