A continuous walking cutting and harvesting robot and its operation method

Abstract

The present invention discloses a continuously walking cutting and harvesting robot and its operation method. The robot comprises a cutting picking module, a floating fruit-grabbing module, a picking path planning and control module, a depth camera, an image processor, an autonomous walking chassis module, a travel speed control module, and a picking point travel error compensation module. The floating fruit-grabbing module, the cutting picking module, and the depth camera are sequentially mounted along the longitudinal centerline of the autonomous walking chassis module at the front, middle, and rear ends. The picking path planning and control module has a built-in picking motion counter for counting the parity of the number of robot arm movements and adjusting the robot arm accordingly. The depth camera is connected to the image processor for signal communication. The travel speed control module adjusts the travel speed of the autonomous walking chassis module according to the fruit distribution density. The picking point travel error compensation module controls the robot arm to accurately locate the spatial coordinates of the fruit stem picking point while the robot is in continuous walking mode. The present invention is used to achieve continuous walking and high-speed harvesting of trellis grapes.

Description

Technical Field

[0001] The present invention relates to the fields of intelligent agricultural equipment and robots, and in particular to a robotic continuous walking cutting and harvesting robot for trellis grapes and an operating method thereof. Background Art

[0002] Harvesting robot technology has rapidly developed both domestically and internationally in recent years, but low harvesting efficiency has become a bottleneck. Conventional harvesting robot solutions require the chassis to be stationary in order to drive the visual and end-user capabilities to accurately locate and pick delicate fruit stems. The precise visual-mechanical positioning requirements, the long round-trip path of the robotic arm to transfer and place the boxes, and the chassis's stationary state while waiting for harvesting to complete consume significant time, hindering conventional harvesting robot solutions from significantly improving harvesting efficiency. Continuous walking harvesting by robots is widely used in the mechanized harvesting of crops such as grains and vegetables. However, in horizontal trellis cultivation, complex conditions such as undulating trellis surfaces and varying heights and positions of grapes hanging on them create significant visual positioning errors during the robot's continuous walking state, making continuous walking harvesting impossible. This results in prolonged waiting times for the robot to remain stationary. Summary of the Invention

[0003] In view of the shortcomings in the prior art, the present invention provides a continuously walking cutting and harvesting robot and an operating method thereof, which enable the harvesting robot to continuously walk and harvest fresh grapes on a trellis at high speed.

[0004] The present invention achieves the above technical objectives through the following technical means.

[0005] A continuously walking cutting and harvesting robot comprising:

[0006] The cutting and harvesting path planning subsystem includes a cutting picking module, a floating fruit receiving module, and a picking path planning control module. The floating fruit receiving module and the cutting picking module are fixedly installed on the front and middle parts of the autonomous walking chassis module along the longitudinal centerline thereof. The picking path planning control module has a built-in picking motion counter, which counts the parity of the number of movements of the cutting picking module's mechanical arm through the picking motion counter and then adjusts the movement direction of the mechanical arm.

[0007] The hand-eye movement-photographing timing coordinated control subsystem includes a depth camera and an image processor, wherein the depth camera is connected to the image processor signal; the depth camera is horizontally mounted on a camera bracket along the longitudinal centerline of the rear end of the autonomous walking chassis module;

[0008] The picking path point walking error compensation subsystem includes an autonomous walking chassis module, a walking speed control module, and a picking point walking error compensation module. The walking speed control module adjusts the walking speed of the autonomous walking chassis module according to the fruit distribution density. The picking point walking error compensation module controls the robotic arm to accurately locate the spatial coordinates of the fruit stem picking point while the robot is in continuous walking state.

[0009] The robotic arm, picking path planning control module, walking speed control module and picking point walking error compensation module all communicate with the main controller in a two-way manner; the actions of the robotic arm, autonomous walking chassis module and lifting platform are controlled by the main controller.

[0010] In the above technical solution, the cutting picking module includes a robotic arm, a cutting disc knife end effector and a robotic arm base. The robotic arm base is fixed above the autonomous walking chassis module, the robotic arm is fixed above the robotic arm base, and the cutting disc knife end effector is fixed at the end wrist of the robotic arm and the working posture is always horizontal.

[0011] In the above technical solution, the floating fruit receiving module includes a fruit box and a lifting platform, the lifting platform is fixed above the autonomous walking chassis module, and the fruit box is detachably connected above the lifting platform.

[0012] An operating method of a continuously walking cutting and harvesting robot:

[0013] The autonomous walking chassis module aligns longitudinally with the row of fruit trees and stops, waiting for the walking command. The end of the robotic arm moves to the right boundary endpoint A and waits for the picking action command. The base of the fruit box rises to a height of h1 from the ground and waits for the floating fruit harvesting command.

[0014] The depth camera continuously detects the hand-eye combined 3D region of interest and autonomously selects between the picking target search mode and the continuous walking harvesting mode based on the actual distribution of fruits in the hand-eye combined 3D region of interest. The number of grapes k in the hand-eye combined 3D region of interest is then transmitted to the walking speed control module, thereby regulating the speed v of the autonomous walking chassis module walking uniformly along the fruit tree row line. k ;

[0015] When the robot is in a continuous walking harvesting state, the image processor combines the hand-eye combination with the spatial coordinates S1, S2, ... S of the picking points of multiple target grapes in the three-dimensional area of ​​interest. i ……S n The data is sent to the main controller, which controls the robotic arm to plan a fixed-width cyclic picking path for all grapes recognized in the hand-eye combined 3D region of interest between the two boundary endpoints A and B at the edge of the working width, which are mirror-symmetrical with respect to the robot's central axis. The picking motion counter calculates the parity of the data.

[0016] Based on the fixed width cyclic picking path planning, the picking point walking error compensation module calculates the spatial coordinates S of each picking path point during continuous walking. i The error is compensated and the coordinates of each picking point S i ′ is transmitted to the main controller, which controls the robotic arm to execute the picking action of accurately locating the picking point in the continuous walking state, and adjusts the operating height of the fruit box according to the height value of the picking point, so as to harvest the fruit without damage right below the three-dimensional area of ​​interest of the hand-eye combination.

[0017] Furthermore, the autonomous selection between the picking target search mode and the continuous walking harvesting mode is made according to the actual distribution of fruits in the three-dimensional area of ​​interest of the hand-eye combination, specifically:

[0018] When the continuously walking cutting and harvesting robot is in the picking target search state, the depth camera moves forward with the robot, and the main controller controls the depth camera to continuously detect the hand-eye combination 3D interest region. When it is recognized that there are grapes entering the hand-eye combination 3D interest region, the image processor obtains the spatial coordinates S of the picking point of each bunch of grapes in the hand-eye combination 3D interest region in real time. i The main controller extracts the spatial coordinate depth value y of the picking point closest to the robot in the depth direction min ; When the depth value y min ≤900mm, control the depth camera to capture a single frame image at the current moment, and immediately turn off the continuous detection function of the depth camera. The main controller calculates the spatial coordinates S of all grape picking points based on the frame image. i , controlling the robotic arm to pick; thus, the switching from the picking target search mode to the continuous walking harvesting mode is realized;

[0019] When the continuously walking splicing harvesting robot is in the continuous walking harvesting condition, the parity change of the data of the picking motion counter triggers the photo signal of the depth camera. When no grapes are located in the three-dimensional interest area of ​​the hand-eye combination in the last frame image obtained by the depth camera, the main controller triggers the continuous detection function of the depth camera, and the robotic arm immediately resets to the right boundary endpoint A to wait for the picking action instruction, realizing the switching from the continuous walking harvesting condition to the picking target search condition.

[0020] Furthermore, the fixed width cyclic picking path planning is specifically as follows: the main controller extracts the horizontal width direction component x of the spatial coordinates of all picking points iThen, a one-way picking path is planned for all picking targets, either from left to right or from right to left. After compensation by the picking point walking error compensation module, the main controller controls the robot arm to be located between the two boundary endpoints A and B, which are mirror-symmetrical relative to the robot's central axis at the edge of the robot's working width, and cyclically executes the picking action with a fixed action width; where: point A is the right boundary endpoint of the harvesting robot, and point B is the left boundary endpoint of the harvesting robot;

[0021] The picking motion counter starts from 0 and counts the number of movements of the robot arm between the two boundary endpoints A and B. When the number of movements is an odd number, the picking direction of the robot arm is from right to left, and the corresponding picking path is A→S1→S2→…→S m →B; When the number of actions is even, the picking direction of the robot arm is from left to right, and the corresponding picking path is B→S m+1 →S m+2 →……→S n-1 →S n →A; where m represents the cumulative number of grapes picked by the robot arm during the picking action from the right boundary endpoint A to the left boundary endpoint B, and n represents the cumulative number of grapes picked by the robot arm during the picking action from the right boundary endpoint A to the left boundary endpoint B and then back to the right boundary endpoint A, and n>m.

[0022] Furthermore, the spatial coordinates of the right boundary endpoint A and the left boundary endpoint B are both located in the space base coordinate system of the robot arm, A=(x A ,y A , z A ), B=(x B ,y B , z B )satisfy:

[0023]

[0024] Wherein: B, H, and D represent the operating width range, operating height range, and operating depth range of the continuously walking cutting and harvesting robot, respectively.

[0025] Furthermore, the autonomous walking chassis module moves at a uniform speed along the fruit tree line. k satisfy:

[0026]

[0027] Where: v k1 >v k2 >v k3 >0.

[0028] Furthermore, the picking point walking error compensation module calculates the spatial coordinates S of each picking path point during continuous walking. iThe error is compensated as follows:

[0029] The picking point walking error compensation module calculates the displacement ΔS of the autonomous walking chassis module's walking mileage relative to the ground absolute coordinate system i =(Δx i , Δy i , Δz i ), as the walking error compensation amount of each picking path point, and obtain the compensated spatial coordinates S of each picking point in the picking path i ′(x i ′,y i ′,z i ′), S i ' is fed back to the main controller to control the robot arm to perform the picking action; the compensated spatial coordinates of each picking point S i 'satisfy:

[0030]

[0031] Where: t i Indicates the spatial coordinates S of each picking point reached by the robotic arm i The time required, v kx 、v ky and v kz are the velocity components of the autonomous walking chassis module along the working width x, depth y and height z directions of the harvesting robot respectively;

[0032] The robotic arm reaches the spatial coordinates S of each picking point i The time required t i satisfy:

[0033]

[0034] Where: M i represents the Euclidean distance between adjacent picking path points, and:

[0035]

[0036] Furthermore, the hand-eye combined three-dimensional region of interest is the largest inscribed cuboid region formed by a common three-dimensional region jointly determined by the working space of the robotic arm and the field of view space of the depth camera.

[0037] The beneficial effects of the present invention are:

[0038] (1) The present invention realizes continuous walking harvesting of fresh grapes on a horizontal trellis through the coordinated cooperation of a cutting picking module, a floating fruit receiving module and an autonomous walking chassis module. The structure is simple, reliable and highly practical.

[0039] (2) The present invention provides a method for splicing and harvesting grapes by continuously walking trellis grape harvesting robots. The method adopts a fixed-width cyclic picking path planning sub-method to plan a one-way picking path for picking targets with similar fruit heights, effectively avoiding the situation where the robotic arm takes a long detour or goes back during the picking process. The hand-eye action-photographing timing coordinated control sub-method coordinates the parity check of the number of robotic arm movements with the image photographing timing of the depth camera, effectively solving the two major problems of being blocked by the robotic arm when taking pictures and the robotic arm waiting for too long for the camera to take pictures. The picking point walking error compensation sub-method uses the displacement of the autonomous walking chassis module from the moment the depth camera takes pictures as the walking error compensation amount of each picking path point, effectively eliminating the dynamic positioning accuracy of the robotic arm in the continuous walking state of the robot and the positioning error caused by the spatiotemporal inconsistency of the picking point coordinates of the visual system statically positioning the single-frame image. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of the continuously moving cutting and harvesting robot of the present invention;

[0041] Figure 2 A schematic diagram of constructing a three-dimensional region of interest for hand-eye combination of limited bunch grape harvesting with multi-parameter constraints according to the present invention;

[0042] Figure 3 Schematic diagram of the multi-hand-eye action-photographing timing coordinated control sub-method of the present invention;

[0043] Figure 4 Schematic diagram of the multi-picking point walking error compensation sub-method of the present invention;

[0044] Figure 5 This is a schematic diagram of a multi-rack inter-row harvesting scenario according to the present invention;

[0045] Figure 6 Schematic diagram of communication between the main controller and other components of the present invention;

[0046] In the figure: 1-cutting picking module, 2-floating fruit receiving module, 3-autonomous walking chassis module, 4-robotic arm, 5-cutting disc knife end effector, 6-robotic arm base, 7-fruit box, 8-lifting platform, 9-depth camera, 10-image processor, 11-camera bracket, 12-picking path planning control module, 13-walking speed control module, 14-picking point walking error compensation module, 15-main controller, 16-picking motion counter, 17-hand-eye combination three-dimensional area of ​​interest, 18-trellis column, 19-trellis net, 20-grape bunch, 21-grape stalk. DETAILED DESCRIPTION

[0047] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0048] The invention provides a continuously walking splicing and harvesting robot comprising a splicing and harvesting path planning subsystem, a hand-eye action-photographing timing coordinated control subsystem and a picking path point walking error compensation subsystem.

[0049] like Figure 1 、 5 As shown, the cutting and harvesting path planning subsystem includes a cutting-type picking module 1, a floating fruit-grabbing module 2, and a picking path planning control module 12. The floating fruit-grabbing module 2 and the cutting-type picking module 1 are fixedly installed at the front and middle parts of the autonomous walking chassis module 3 along the longitudinal centerline of the autonomous walking chassis module 3, and are both located above the autonomous walking chassis module 3. The cutting-type picking module 1 includes a robotic arm 4, a cutting disc end effector 5, and a robotic arm base 6. The robotic arm base 6 is fixed above the autonomous walking chassis module 3, the robotic arm 4 is fixed above the robotic arm base 6, and the cutting disc end effector 5 is fixed to the end wrist of the robotic arm 4, and the working posture is always horizontal.

[0050] like Figure 2 As shown, the floating fruit receiving module 2 includes a fruit box 7 and a lifting platform 8. The lifting platform 8 is fixed above the autonomous walking chassis module 3. The fruit box 7 is detachably connected above the lifting platform 8, so that it can be replaced after being filled with grapes.

[0051] The picking path planning control module 12 has a built-in picking motion counter 16, which counts the parity of the number of movements of the robotic arm 4 through the picking motion counter 16, and then the main controller 15 adjusts the movement direction of the robotic arm 4, and has the function of quickly planning the picking motion path of the cutting picking module 1.

[0052] The hand-eye movement-photographing timing coordinated control subsystem includes a depth camera 9 and an image processor 10. The depth camera 9 is connected to the image processor 10 by signal. The depth camera 9 outputs image signals to the image processor 10 unidirectionally. The image processor 10 outputs signals to the main controller 15 unidirectionally. The depth camera 9 is controlled by the unidirectional signal of the main controller 15. The depth camera 9 is horizontally installed on the camera bracket 11 along the longitudinal center line of the tail end of the autonomous walking chassis module 3. The camera bracket 11 is fixed above the autonomous walking chassis module 3.

[0053] The picking point walking error compensation subsystem includes an autonomous walking chassis module 3, a walking speed control module 13 and a picking point walking error compensation module 14. The walking speed control module 13 has the function of adjusting the walking speed of the autonomous walking chassis module 3 according to the fruit distribution density. The picking point walking error compensation module 14 can control the robotic arm 4 to accurately locate the spatial coordinates of the fruit stalk picking point when the robot is in a continuous walking state.

[0054] like Figure 6 As shown, the robotic arm 4, picking path planning control module 12, walking speed control module 13, and picking point walking error compensation module 14 all communicate bidirectionally with the main controller 15. The movements of the robotic arm 4, autonomous walking chassis module 3, and lifting platform 8 are controlled by the main controller 15.

[0055] The continuously walking cutting harvesting robot can realize the picking target search mode and the continuous walking harvesting mode. The method for judging the picking target search mode is as follows: during the continuous walking process of the robot, the depth camera 9 does not detect the existence of grapes in the hand-eye combined three-dimensional interest area 17, and continues to detect; the method for judging the continuous walking harvesting mode is as follows: during the continuous walking process of the robot, the depth camera 9 detects the existence of grapes in the hand-eye combined three-dimensional interest area 17, and the depth value y of the spatial coordinate of the target grape picking point closest to the robot in the depth direction is min ≤900mm, the picking point spatial coordinates are located in the space base coordinate system of the robotic arm 4.

[0056] like Figure 2 As shown, the hand-eye combined 3D region of interest 17 is the largest inscribed cuboid region formed by the common 3D region jointly determined by the workspace Θ of the robot arm 4 and the field of view space θ of the depth camera 9. All grapes within the hand-eye combined 3D region of interest 17 can be correctly identified by the depth camera 9 and picked by the cutting-type picking module 1. The hand-eye combined 3D region of interest 17, the workspace Θ of the robot arm 4, and the field of view space θ of the depth camera 9 satisfy:

[0057] Ψ=[B,H,D](Ψ∈Θ∩θ)

[0058]

[0059]

[0060] Where: Ψ represents the hand-eye combined three-dimensional interest region 17, B, H, and D represent the operating width range, operating height range, and operating depth range of the continuously walking cutting and harvesting robot, respectively, and W b is the working width of the working space Θ of the robot arm 4, W g is the horizontal width of the fruit box body, D b D is the operating depth range that the robot arm 4 needs to meet, min The minimum detection depth D is set for the vertical distance between the depth camera 9 and the rear plane M1 of the fruit box 7. max The maximum detection depth set by the depth camera 9 to the vertical distance between a plane M2 inside the fruit box 7 and parallel to M1, H s H is the installation height of the upper surface of the robot arm base 6 relative to the ground, bmaxH is the maximum operating height of the robot arm. bmin is the minimum operating height reached by the robot body, J is the change in picking height under the same scaffolding net surface, θ w is the horizontal angle of the field of view of the depth camera 9 in the horizontal plane, θ v is the longitudinal angle of the field of view space θ of the depth camera 9 in the vertical plane, H c is the installation height of the depth camera 9 relative to the ground, W emin is the minimum detection depth D of the depth camera 9 min The corresponding field of view width range, H emin is the minimum detection depth D of the depth camera 9 min The corresponding field of view height range is shown below.

[0061] The depth camera 9 is installed at a height H relative to the ground. c Take the following values:

[0062]

[0063]

[0064] In this embodiment, B is 1000 mm, H is 1750-2000 mm, D is 600-1000 mm, and H is c Take 1750mm.

[0065] The present invention provides an operation method of a continuously walking splicing and harvesting robot, which includes a fixed-width cyclic picking path planning sub-method, a hand-eye action-photographing timing coordinated control sub-method, a picking point walking error compensation sub-method, and a splicing and harvesting operation process of the continuously walking trellis grape harvesting robot.

[0066] Fixed width cyclic picking path planning sub-method: When the main controller 15 receives the spatial coordinates S of all picking points in the hand-eye combined three-dimensional interest area 17 calculated by the image processor 10 i =(x i ,y i , z i ), the main controller 15 immediately extracts the horizontal width direction component x of the spatial coordinates of all picking points i, then a one-way picking path is planned for all picking targets from left to right or from right to left. After compensation by the picking point walking error compensation module 14, the main controller 15 controls the robot arm 4 to be located between the two boundary endpoints A and B that are mirror-symmetrical relative to the robot's central axis at the edge of the robot's working width, and cyclically executes the picking action with a fixed action width; wherein: point A is the right boundary endpoint of the harvesting robot, point B is the left boundary endpoint of the harvesting robot, and the picking motion counter 16 starts counting the number of actions of the robot arm 4 between the two boundary endpoints A and B from 0; when the number of actions is an odd number, the picking action direction of the robot arm 4 is from right to left, and the corresponding picking path is A→S1→S2→……→S m →B(x A >x1>x2>……>x m >x B ); When the number of actions is even, the picking action direction of the robot arm 4 is from left to right, and the corresponding picking path is B→S m+1 →S m+2 →……→S n-1 →S n →A(x A >x n >x n-1 >……>x m+2 >x m+1 >x B ); wherein, m represents the cumulative number of grapes picked by the robot arm 4 during the picking action from the right boundary endpoint A to the left boundary endpoint B, and n represents the cumulative number of grapes picked by the robot arm 4 during the picking action from the right boundary endpoint A to the left boundary endpoint B and then back to the right boundary endpoint A (i.e., one picking cycle), and n>m.

[0067] The specific workflow of the hand-eye movement-photographing timing coordinated control sub-method is as follows: Figure 3 shown.

[0068] When the continuously walking cutting and harvesting robot is in the picking target search state, the specific workflow of the hand-eye action-photography timing coordinated control sub-method is as follows:

[0069] The main controller 15 controls the depth camera 9 to continuously detect the hand-eye combination three-dimensional interest area 17, determines whether there are grapes in the hand-eye combination three-dimensional interest area 17, and controls the robot arm 4 to reach the boundary endpoint A or B of the field of view that does not block the depth camera 9 in the hand-eye combination three-dimensional interest area 17 to wait for the picking action instruction, where: the spatial coordinates of the two points A and B are both located in the spatial base coordinate system of the robot arm 4, A=(x A ,y A , z A ), B=(x B ,y B , z B)The two boundary endpoints satisfy:

[0070]

[0071] In this embodiment, the two boundary endpoints A=(600, 800, 1850) and B=(-600, 800, 1850).

[0072] When the continuously walking cutting and harvesting robot is in the continuous walking harvesting state, the specific workflow of the hand-eye action-photographing timing coordinated control sub-method is as follows:

[0073] The main controller 15 coordinates and controls the photo-taking timing of the depth camera 9 according to the action execution status of the robot arm 4 (reaching point A or point B). When the data parity of the picking motion counter 16 changes, the main controller 15 triggers the photo-taking signal of the depth camera 9 and sends the obtained single-frame image to the image processor 10 to refresh and overwrite the previous frame image. Then, the image processor 10 calculates the spatial coordinates S of the picking points of the target grape bunches based on the refreshed single-frame image. i (The specific calculation method is the existing technology), and the spatial coordinates S of the picking points of multiple target grapes are i The main controller 15 sends the spatial coordinates S of the target grape picking points to the target grape picking points. i The information is sent in sequence to the picking path planning control module 12, the walking speed control module 13 and the picking point walking error compensation module 14 to control the cutting picking module 1 and the autonomous walking chassis module 3 to perform continuous walking harvesting in coordinated action sequence.

[0074] When a continuously walking cutting and harvesting robot needs to switch between the picking target search mode and the continuous walking harvesting mode, the specific workflow of the hand-eye action-photography timing coordinated control sub-method is as follows:

[0075] When the continuously moving cutting and harvesting robot is in the picking target search mode, the depth camera 9 moves forward with the robot, and the main controller 15 controls the depth camera 9 to continuously detect the hand-eye combination three-dimensional interest area 17. When it is recognized that grapes enter the hand-eye combination three-dimensional interest area 17, the image processor 10 obtains the spatial coordinates S of the picking point of each bunch of grapes in the hand-eye combination three-dimensional interest area 17 in real time. i =(x i ,y i , z i ), the main controller 15 extracts the spatial coordinate depth value y of the picking point closest to the robot in the depth direction min ; When the depth value y min ≤900mm, the main controller 15 controls the depth camera 9 to capture a single frame image at the current moment, and immediately turns off the continuous detection function of the depth camera 9. The main controller 15 calculates the spatial coordinates S of all grape picking points based on the frame image.i , controlling the robotic arm 4 to perform picking; thereby, switching from the picking target search mode to the continuous walking harvesting mode is achieved. When the continuously walking splicing harvesting robot is in the continuous walking harvesting mode, the change in the parity of the data from the picking motion counter 16 triggers the photo signal of the depth camera 9. When the last frame of the image obtained by the depth camera 9 shows that no grapes are within the hand-eye combined three-dimensional area of ​​interest 17, the main controller 15 triggers the continuous detection function of the depth camera 9, and the robotic arm 4 immediately resets to point A to await the picking action instruction, thus achieving the switch from the continuous walking harvesting mode to the picking target search mode.

[0076] The sub-method for compensating the walking error of the picking point is as follows: Figure 4 As shown, it includes a walking error compensation algorithm for the picking point and a walking speed control algorithm based on the current fruit density.

[0077] The picking point walking error compensation algorithm is only executed when the continuously walking cutting harvesting robot is in the continuous walking harvesting working condition. The picking point walking error compensation module 14 calculates the displacement ΔS of the autonomous walking chassis module 3 walking mileage relative to the ground absolute coordinate system. i =(Δx i , Δy i , Δz i ), as the walking error compensation amount of each picking path point, and obtain the compensated spatial coordinates S of each picking point in the picking path i ′(x i ′,y i ′,z i ′), S i ' is fed back to the main controller 15, which controls the robot arm 4 to perform the picking action, S i 'satisfy:

[0078]

[0079] Where: t i Indicates that the robot arm 4 reaches the spatial coordinates S of each picking point i The time required, t i According to the Euclidean distance M between adjacent picking path points i and the end linear velocity v of the robot arm 4 s Calculated, the Euclidean distance M between adjacent picking path points i , the robotic arm 4 reaches the spatial coordinates S of each grape picking point i The time required t i satisfy:

[0080]

[0081]

[0082] Where: The end linear velocity v of the robot arm 4 s When there is no big sudden change in the direction of motion, it can be regarded as uniform speed, v kx 、v ky and v kz are the velocity components of the autonomous walking chassis module 3 along the harvesting robot's operating width x, depth y, and height z directions; in this embodiment, v k =v kx 、v ky =0, v kz =0,v s =600mm / s.

[0083] The walking speed control algorithm based on the current fruit density adaptively adjusts the speed v of the autonomous walking chassis module 3 according to the number of grapes in the current hand-eye combined three-dimensional interest area 17. k The robot can speed up to quickly pass through sparse fruits and slow down to carefully harvest dense fruits, thereby improving the efficiency of continuous walking and cutting harvesting. The main controller 15 obtains the number of grapes in the hand-eye combined three-dimensional interest area 17 calculated by the image processor 10 in real time and transmits it to the walking speed control module 13. The walking speed control module 13 calculates the walking speed v of the autonomous walking chassis module 3 based on the number of grapes. k , and then transmit it to the main controller 15 to realize the control of the walking speed of the autonomous walking chassis module 3; the walking speed v of the autonomous walking chassis module 3 k satisfy:

[0084]

[0085] Where: v k1 >v k2 >v k3 >0. In this embodiment, v k1 =200mm / s, v k2 =150mm / s, v k3 =100mm / s.

[0086] The scene of harvesting between rows of trellises is shown as follows: Figure 5 As shown in the figure, the continuous walking cutting and harvesting process of the trellis grape harvesting robot is as follows:

[0087] The main controller 15 controls the autonomous walking chassis module 3 to align longitudinally with the row of fruit trees and then stop and wait for the walking instruction. The main controller 15 controls the end of the robotic arm 4 to move to the right boundary endpoint A and wait for the picking action instruction. The main controller 15 controls the lifting platform 8 to raise the base of the fruit box 7 to a height h1 above the ground and wait for the floating fruit harvesting instruction. In this embodiment, h1 = 1250 mm.

[0088] The main controller 15 controls the depth camera 9 to continuously detect the hand-eye combination three-dimensional area of ​​interest 17, and autonomously selects between the picking target search mode and the continuous walking harvesting mode based on the actual distribution of fruits in the hand-eye combination three-dimensional area of ​​interest 17. The main controller 15 then transmits the number k of grapes in the hand-eye combination three-dimensional area of ​​interest 17 to the walking speed control module 13. The walking speed control module 13 controls the speed v of the autonomous walking chassis module 3 walking uniformly along the fruit tree row line based on the number k of grapes in the hand-eye combination three-dimensional area of ​​interest 17. k ;

[0089] When the robot is in a continuous walking harvesting state, the image processor 10 combines the hand-eye combination with the spatial coordinates S1, S2, ..., S of the picking points of multiple target grape bunches in the three-dimensional area of ​​interest 17. i ……S n The data is sent to the main controller 15, which controls the robot arm 4 to plan a fixed-width cyclic picking path for all grapes identified in the hand-eye combined three-dimensional area of ​​interest 17 between the two boundary endpoints A and B that are mirror-symmetrical with respect to the robot's central axis at the edge of the working width, based on the parity of the data from the picking motion counter 16;

[0090] The main controller 15 transmits the fixed width cycle picking path planning data to the picking point walking error compensation module 14, and the picking point walking error compensation module 14 calculates the spatial coordinates S of each picking path point during continuous walking. i The error is compensated and the coordinates of each picking point S i ' is transmitted to the main controller 15, which controls the robotic arm 4 to pull the cutting disc end effector 5 with a certain tolerance to perform the picking action of accurately locating the picking point in a continuous walking state. At the same time, the main controller 15 adjusts the operating height of the fruit box 7 according to the depth value of the spatial coordinate of the picking point, and harvests the fruit without damage directly below the hand-eye combined three-dimensional area of ​​interest 17;

[0091] At this point, the harvesting robot completes the continuous walking, splicing and harvesting operation of the trellis grapes in the continuous walking state of the autonomous walking chassis module 3.

[0092] The embodiments described are preferred implementations of the present invention, but the present invention is not limited to the above implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention are within the scope of protection of the present invention.

Claims

1. A method for operating a continuously walking cutting and harvesting robot, characterized in that: The continuously walking cutting and harvesting robot comprises: A cutting and harvesting path planning subsystem comprises a cutting picking module (1), a floating fruit receiving module (2) and a picking path planning control module (12), wherein the floating fruit receiving module (2) and the cutting picking module (1) are fixedly mounted on the front end and the middle part of the autonomous walking chassis module (3) along the longitudinal center line thereof, and the picking path planning control module (12) has a built-in picking motion counter (16), which counts the odd and even number of motions of the mechanical arm (4) of the cutting picking module (1) through the picking motion counter (16), thereby adjusting the motion direction of the mechanical arm (4); A hand-eye movement-photographing timing coordinated control subsystem includes a depth camera (9) and an image processor (10), wherein the depth camera (9) is signal-connected to the image processor (10); the depth camera (9) is horizontally mounted on a camera bracket (11) along the longitudinal centerline of the rear end of the autonomous walking chassis module (3); A picking path point walking error compensation subsystem comprises an autonomous walking chassis module (3), a walking speed control module (13) and a picking point walking error compensation module (14), wherein the walking speed control module (13) adjusts the walking speed of the autonomous walking chassis module (3) according to the fruit distribution density, and the picking point walking error compensation module (14) controls the robot arm (4) to accurately locate the spatial coordinates of the fruit stem picking point when the robot is in a continuous walking state; The mechanical arm (4), the picking path planning control module (12), the walking speed control module (13), and the picking point walking error compensation module (14) all communicate bidirectionally with the main controller (15); the movements of the mechanical arm (4), the autonomous walking chassis module (3), and the lifting platform (8) are controlled by the main controller (15); The operation method is: The autonomous walking chassis module (3) is aligned longitudinally with the fruit tree row line and then stops and waits for a walking instruction. The end of the robotic arm (4) moves to the right boundary endpoint A and waits for a picking action instruction. The base of the fruit box (7) rises to a height h1 from the ground and waits for a floating fruit harvesting instruction. The depth camera (9) continuously detects the hand-eye combination three-dimensional interest region (17), and autonomously selects between the picking target search mode and the continuous walking harvesting mode according to the actual distribution of the fruits in the hand-eye combination three-dimensional interest region (17). Then, the number k of grapes in the hand-eye combination three-dimensional interest region (17) is transmitted to the walking speed control module (13), and then the speed v of the autonomous walking chassis module (3) walking uniformly along the fruit tree line is controlled. k ; When the robot is in a continuous walking harvesting state, the image processor (10) combines the hand and eye to obtain the spatial coordinates S1, S2, ... S of the picking points of multiple target grapes in the three-dimensional area of ​​interest (17). i ……S n The data is sent to the main controller (15), and the main controller (15) controls the robot arm (4) to perform fixed-width cyclic picking path planning for all grapes identified in the hand-eye combined three-dimensional interest area (17) between the two boundary endpoints A and B that are mirror-symmetrical relative to the robot's central axis at the edge of the working width according to the parity of the data counted by the picking motion counter (16); Based on the fixed width cyclic picking path planning, the picking point walking error compensation module (14) calculates the spatial coordinates S of each picking path point during continuous walking. i The error is compensated and the coordinates of each picking point S i ′ The information is transmitted to the main controller (15), which controls the robotic arm (4) to execute the picking action of accurately locating the picking point in a continuous walking state, and at the same time adjusts the operating height of the fruit box (7) according to the depth value of the spatial coordinate of the picking point, so as to harvest the fruit without damage right below the three-dimensional area of ​​interest (17) of the hand-eye combination.

2. The operation method according to claim 1, characterized in that: The autonomous selection between the picking target search working condition and the continuous walking harvesting working condition is made according to the actual distribution of fruits in the hand-eye combination three-dimensional interest area (17), specifically: When the continuously moving cutting and harvesting robot is in a picking target searching state, the depth camera (9) moves forward with the robot, and the main controller (15) controls the depth camera (9) to continuously detect the hand-eye combination three-dimensional interest region (17). When it is recognized that grapes enter the hand-eye combination three-dimensional interest region (17), the image processor (10) obtains the picking point spatial coordinates S of each bunch of grapes in the hand-eye combination three-dimensional interest region (17) in real time. i The main controller (15) extracts the spatial coordinate depth value y of the picking point closest to the robot in the depth direction min ; When the depth value y min ≤900mm, control the depth camera (9) to capture a single frame image at the current moment, and immediately turn off the continuous detection function of the depth camera (9), and the main controller (15) calculates the spatial coordinates S of all grape picking points according to the frame image. i , controlling the robotic arm (4) to perform picking; thereby, switching from a picking target search mode to a continuous walking harvesting mode is achieved; When the continuously walking splicing harvesting robot is in a continuous walking harvesting mode, the parity change of the data of the picking motion counter (16) triggers the photo signal of the depth camera (9). When no grapes are located in the hand-eye combined three-dimensional interest area (17) in the last frame image obtained by the depth camera (9), the main controller (15) triggers the continuous detection function of the depth camera (9), and the robot arm (4) immediately resets to the right boundary endpoint A to wait for the picking action instruction, thereby realizing the switching from the continuous walking harvesting mode to the picking target search mode.

3. The operation method according to claim 2, characterized in that: The fixed width cyclic picking path planning is specifically as follows: the main controller (15) extracts the horizontal width direction component x of the spatial coordinates of all picking points i Then, a unidirectional picking path is planned for all picking targets from left to right or from right to left. After compensation by the picking point walking error compensation module (14), the main controller (15) controls the robot arm (4) to be located between the two boundary endpoints A and B that are mirror-symmetrical with respect to the robot's central axis at the edge of the robot's working width, and cyclically executes the picking action with a fixed action width; wherein: point A is the right boundary endpoint of the harvesting robot, and point B is the left boundary endpoint of the harvesting robot; The picking motion counter (16) starts counting the number of movements of the robot arm (4) between the two boundary endpoints A and B from 0; when the number of movements is an odd number, the picking motion direction of the robot arm (4) is from right to left, and the corresponding picking path is A→S1→S2→……→S m →B; When the number of actions is even, the picking direction of the robot arm (4) is from left to right, and the corresponding picking path is B→S m+1 →S m+2 →……→S n-1 →S n →A; wherein, m represents the cumulative number of grapes picked by the robot arm (4) during the picking action from the right boundary endpoint A to the left boundary endpoint B, and n represents the cumulative number of grapes picked by the robot arm (4) during the picking action from the right boundary endpoint A to the left boundary endpoint B and then back to the right boundary endpoint A, and n>m.

4. The operation method according to claim 3, characterized in that: The spatial coordinates of the right boundary endpoint A and the left boundary endpoint B are both located in the space base coordinate system of the robot arm (4), A=(x A ,y A , z A ), B=(x B ,y B , z B )satisfy: Wherein: B, H, and D represent the operating width range, operating height range, and operating depth range of the continuously walking cutting and harvesting robot, respectively.

5. The operation method according to claim 1, characterized in that: The speed v of the autonomous walking chassis module (3) walking uniformly along the fruit tree line k satisfy: Where: v k1 >v k2 >v k3 >0.

6. The operation method according to claim 1, characterized in that: The picking point walking error compensation module (14) calculates the spatial coordinates S of each picking path point during continuous walking. i The error is compensated as follows: The picking point walking error compensation module (14) calculates the displacement ΔS of the autonomous walking chassis module (3) walking mileage relative to the ground absolute coordinate system i =(Δx i , Δy i , Δz i ), as the walking error compensation amount of each picking path point, and obtain the compensated spatial coordinates S of each picking point in the picking path i ′ (x i ′ ,y i ′ , z i ′ ), S i ′ Feedback is sent to the main controller (15) to control the mechanical arm (4) to perform the picking action; the compensated spatial coordinates S of each picking point i ′ satisfy: Where: t i Indicates that the robotic arm (4) reaches the spatial coordinates S of each picking point i The time required, v kx 、v ky and v kz are the velocity components of the autonomous walking chassis module (3) along the working width x, depth y and height z directions of the harvesting robot respectively; The robotic arm (4) reaches the spatial coordinates S of each picking point i The time required t i satisfy: Where: M i represents the Euclidean distance between adjacent picking path points, and:

7. The operation method according to claim 4, characterized in that: The hand-eye combined three-dimensional region of interest (17) is the largest inscribed rectangular parallelepiped region formed by a common three-dimensional region jointly determined by the working space of the robotic arm (4) and the field of view space of the depth camera (9).

8. The operation method according to claim 4, characterized in that: The cutting-type picking module (1) comprises a robotic arm (4), a cutting disc knife end effector (5) and a robotic arm base (6); the robotic arm base (6) is fixed above the autonomous walking chassis module (3); the robotic arm (4) is fixed above the robotic arm base (6); the cutting disc knife end effector (5) is fixed to the end wrist of the robotic arm (4) and the working posture is always horizontal.

9. The operation method according to claim 4, characterized in that: The floating fruit receiving module (2) comprises a fruit box (7) and a lifting platform (8), wherein the lifting platform (8) is fixed above the autonomous walking chassis module (3), and the fruit box (7) is detachably connected above the lifting platform (8).

Images