Picking robot and picking method

Abstract

The invention provides a picking robot and a picking method. The picking robot comprises a moving device, a navigation system, a visual system, a mechanical arm, a manipulator, a storage device and a control system, the moving device comprises a rack and a plurality of moving mechanisms, and the moving mechanisms are arranged on the rack; the navigation system is arranged on the rack and used for recognizing the position of the cultivation frame and establishing a map of the target area; the visual system is arranged on the rack and is used for acquiring a complete image of a target fruit and vegetable and three-dimensional position information of the target fruit and vegetable; the mechanical arm is arranged on the rack, and a manipulator is arranged at the free end of the mechanical arm and used for picking target fruits and vegetables; the storage device is arranged on the rack and used for storing the target fruits and vegetables picked by the manipulator; the moving device, the navigation system, the visual system, the mechanical arm, the manipulator and the storage device are in communication connection with the control system. The control system plans a moving route and a picking path and controls the moving device to move, and the mechanical arm and the manipulator carry out picking and store the picked materials in the storage device.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural machinery, and in particular to a picking robot and a picking method. Background Art

[0002] At present, fruit and vegetable picking in my country is still mainly done manually, and there are few agricultural machines suitable for large-scale fruit and vegetable picking. Fruit and vegetable picking is a key link in the growth cycle of fruits and vegetables from the planting site to the table, and its picking efficiency and quality directly affect the quality and economic benefits of fruits and vegetables. Applying intelligent equipment such as industrial robots and mobile robots with a high degree of automation to agriculture will not only help improve picking efficiency and quality, but also promote the transformation of traditional farmers into new-era farmers and the upgrade of traditional agriculture to new agriculture.

[0003] At present, the picking efficiency of most fruit and vegetable picking robots is relatively low, generally lower than manual harvesting, and their feasibility, effectiveness and commercialization are all poor. The robot operation process and operation steps using a five-link robotic arm are relatively complicated, and the picking space occupied is large, which is not suitable for compact fruit and vegetable planting environments. In addition, the visual scheme of existing picking robots has difficulty in identifying obscured strawberries, making it impossible to pick such fruits and vegetables. Therefore, the current picking efficiency and success rate of fruit and vegetable picking robots are significantly lower than those of manual picking. In addition, the capabilities of most existing picking robots are not sufficient to support them in completing the entire strawberry picking process, and they can often only solve part of the problem in the picking process. Summary of the invention

[0004] The present invention provides a picking robot and a picking method, which are used to solve the problem that the picking robot in the prior art has a low degree of automation and cannot realize continuous picking.

[0005] The present invention provides a picking robot, comprising: a mobile device, a navigation system, a visual system, a mechanical arm, a manipulator, a storage device and a control system, wherein the mobile device comprises: a frame and a plurality of mobile mechanisms, wherein the plurality of mobile mechanisms are arranged on the frame; the navigation system is arranged on the frame, and is used to identify the position of a cultivation frame and establish a map of a target area; the visual system is arranged on the frame, and is used to obtain a complete image of target fruits and vegetables and three-dimensional position information of the target fruits and vegetables; the mechanical arm is arranged on the frame, and a manipulator is arranged at a free end of the mechanical arm, and is used to pick target fruits and vegetables; the storage device is arranged on the frame, and is used to store the target fruits and vegetables picked by the manipulator; the mobile device, the navigation system, the visual system, the mechanical arm, the manipulator and the storage device of the control system are respectively connected to the control system for communication; the control system determines a target route based on a target position and a map of a target area; controls the movement of the mobile device based on the target route; determines a picking order based on the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, controls the mechanical arm and the manipulator to move to pick the target fruits and vegetables, and places them in the storage device.

[0006] The present invention also provides a picking method of a picking robot, which is based on the picking robot as described in any one of the above items, and includes: identifying the position of a cultivation rack and establishing a target area map, wherein the target area map includes the position of the cultivation rack and the driving route of the picking robot; based on the initial control of the picking robot, a mobile device moves to the position of the target cultivation rack according to the target route; obtains a complete image of the target fruits and vegetables and three-dimensional position information of the target fruits and vegetables, and determines a picking order; based on the picking order, corresponding to the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, controls the robotic arm to drive the robotic arm to move toward the target fruits and vegetables, and controls the robotic arm to pick the target fruits and vegetables; obtains the current weight of the target fruits and vegetables; based on the current weight, controls the robotic arm to place the target fruit in the target area of ​​a storage device.

[0007] The picking robot and picking method provided by the present invention establish a target fruit and vegetable area map through a navigation system, determine the robot's moving route, and control the movement of the mobile device; obtain the complete image of the target fruit and vegetable and the three-dimensional position information of the target fruit and vegetable through a visual system, determine the picking order, control the mechanical arm to move to the position of the target fruit and vegetable, and pick the target fruit and vegetable through a mechanical hand; based on the position of the storage device, control the movement of the mechanical arm and the mechanical hand, and place the picked target fruit and vegetable in the fruit frame corresponding to the storage device. The present invention adopts an intelligent control system to control all modules, the mobile platform is flexible in action, the navigation system is accurately positioned, the visual system can identify and locate the obstructed fruits and vegetables, the mechanical arm has a large movement space and good obstacle avoidance performance, the mechanical hand can realize soft picking and combine with the storage device for sorting and storage, realize automatic fruit and vegetable picking, and has high picking efficiency, success rate and low damage rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 It is a structural schematic diagram of the picking robot provided by the present invention; Figure 2 This is one of the structural schematic diagrams of the picking robot provided by the present invention when picking fruits and vegetables; Figure 3 This is the second structural schematic diagram of the picking robot provided by the present invention when picking fruits and vegetables; Figure 4 yes Figure 3 A partial enlarged view of the middle A area; Figure 5 This is the third structural schematic diagram of the picking robot provided by the present invention when picking fruits and vegetables; Figure 6 It is a structural schematic diagram of the mechanical arm provided by the present invention; Figure 7 It is a structural schematic diagram of the manipulator provided by the present invention; Figure 8 is a schematic structural diagram of a mobile device provided by the present invention; Fig. 9 is a structural schematic diagram of a battery system provided by the present invention; Fig.10 is a structural schematic diagram of a battery module provided by the present invention; Fig.11 is a bottom view of the battery module provided by the present invention; Fig.12 It is a schematic diagram of the process of the picking method provided by the present invention; Fig.13 It is a schematic diagram of the walking route of the robot provided by the present invention in the target area; Reference numerals: 10. Mobile device; 11. Frame; 12. Mobile mechanism; 121. Adapter; 122. Steering assembly; 123. Shock absorbing assembly; 124. Wheel assembly; 20. Navigation system; 30. Vision system; 40. Robotic arm; 41. Mobile platform; 411. First mobile assembly; 412. Second mobile assembly; 413. Guide rail; 42. Hybrid mechanism; 421. First connecting rod; 422. Second connecting rod; 423. Bracket; 43. Rotating mechanism; 431. First rotating arm; 432. Second rotating arm; 50. Manipulator; 51. Knuckle; 52. Rotating shaft; 53. Locking assembly; 531. Ratchet; 532. Paw; 533. Cable; 534. Elastic member; 54. Flexible cavity; 55. Pipeline; 56. Flexible contact layer; 60. storage device; 61. storage frame; 62. fruit frame; 621. first area; 622. second area; 623. third area; 70. Battery system; 71. First battery module; 72. Second battery module; 73. Mounting frame; 74. Quick release lock; 741. Toggle unit; 75. Handle; 76. Interface; 77. Charging port; 78. Display module; 79. Switch; 80. Control system; 90. Cultivation rack. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] In the description of the embodiments of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0012] In the description of the embodiments of the present invention, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0013] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0014] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0015] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific processes and examples of materials, but those of ordinary skill in the art can be aware of the applicability of other processes and / or the use of other materials.

[0016] Combine the following Figure 1-Figure 12 The harvesting robot and harvesting method of the present invention are described.

[0017] like Figure 1 As shown, the harvesting robot provided by the embodiment of the present invention includes: a mobile device 10, a navigation system 20, a visual system 30, a mechanical arm 40, a mechanical hand 50, a storage device 60 and a control system 80.

[0018] like Figure 8As shown, the mobile device 10 includes a frame 11 and a plurality of mobile mechanisms 12, and the plurality of mobile mechanisms 12 are arranged on the frame 11. In practical applications, the frame 11 is made of high-strength aluminum alloy material to ensure that the structure is lightweight while maintaining sufficient load-bearing capacity and stability. The frame 11 is designed as a square or circular frame, and a reinforcing rib is provided in the center to increase the overall rigidity. A plurality of mounting holes are reserved on the edge of the frame 11 for fixing the plurality of mobile mechanisms 12. Figure 8 As shown, four mobile mechanisms 12 are evenly distributed around the frame 11 to achieve omnidirectional mobility. Each mobile mechanism 12 is independently controlled and works together to achieve complex movements such as forward, backward, and rotation. It should be noted that the omnidirectional mobile device 10 has the ability to travel in all directions, turn while walking, turn in place, move sideways, and turn in place through a narrow space.

[0019] The navigation system 20 is arranged on the frame 11, and is used to identify the position of the cultivation rack 90 and establish a target area map. Specifically, the navigation system 20 can obtain a global map of the target area (such as a picking greenhouse) as the initial basis for the robot, and the map includes the position of the cultivation rack 90. ​​Based on the map, the robot's driving route can be determined, and then the target route can be determined. The navigation system 20 of the present invention has strong real-time performance, is not affected by the position of the cultivation rack 90, has strong adaptability to the planting environment, and has high stability.

[0020] The visual system 30 is arranged on the frame 11, and is used to obtain the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables. The visual system 30 can take pictures of the target fruits and vegetables, fuse the pictures taken, or perform identification and positioning, so as to obtain the complete image of the target fruits and vegetables, which is beneficial for robot picking.

[0021] like Figure 2 and Figure 5 As shown, the robotic arm 40 is disposed on the frame 11, and the robotic arm 40 can rotate and move up and down around the installation position of the frame 11. Adjusting the angle and height of the robotic arm 40 helps the robotic arm 50 connected to the free end of the robotic arm 40 to pick the target fruits and vegetables. In the embodiment of the present invention, there can be two robotic arms 40, and the two robotic arms 40 are disposed on opposite sides of the frame 11. The number of robotic arms 40 in the embodiment of the present invention is not specifically limited, and can also be 1. It should be noted that the number of robotic arms 40 and robotic arms 50 is the same, and each robotic arm 40 and the corresponding robotic arm 50 can be controlled separately.

[0022] The storage device 60 is arranged on the rack 11, such as Figure 4 As shown, it is used to store the target fruits and vegetables picked by the robot 50. Specifically, a fruit frame 62 is placed on the storage device 60, and the target fruits and vegetables picked by the robot 50 can be contained.

[0023] Further, the mobile device 10, the navigation system 20, the visual system 30, the robot arm 40, the robot hand 50 and the storage device 60 are respectively connected to the control system 80 for communication. Specifically, the control system 80 can determine the target route of the robot movement based on the target position and the target area map. The control system 80 can determine the picking order based on the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, and control the robot arm 40 to rotate so that the robot arm 40 moves to the picking position of the target fruits and vegetables; further, the control system 80 controls the robot hand 50 to pick the target fruits and vegetables. After the picking is completed, the control system 80 controls the robot arm 40 to move to the storage position, and controls the robot hand 50 to place the target fruits and vegetables in the fruit frame 62 on the storage device 60, thereby completing the picking of a target fruit and vegetable, and so on, the target fruits and vegetables in the entire area are picked.

[0024] The picking robot provided by the embodiment of the present invention establishes a target fruit and vegetable area map through the navigation system 20, determines the robot's moving route, and controls the movement of the mobile device 10; obtains the complete image of the target fruit and vegetable and the three-dimensional position information of the target fruit and vegetable through the visual system 30, determines the picking order, controls the mechanical arm 40 to move to the position of the target fruit and vegetable, and picks the target fruit and vegetable through the mechanical hand 50; based on the position of the storage device 60, controls the actions of the mechanical arm 40 and the mechanical hand 50, and places the picked target fruit and vegetable in the fruit frame 62 corresponding to the storage device 60. The present invention adopts an intelligent control system to control all modules, the mobile platform is flexible in action, the navigation system is accurate in positioning, the visual system can identify and locate the blocked fruit and vegetable, the mechanical arm has a large movement space and good obstacle avoidance performance, the mechanical hand can realize soft picking and sorting and storage in combination with the storage device, and realizes automatic picking of fruits and vegetables, with high picking efficiency, success rate and low damage rate.

[0025] The robot arm 40 provided in the embodiment of the present invention includes: a mobile platform 41, a hybrid mechanism 42 and a swivel mechanism 43. The mobile platform 41 includes a first mobile component 411, a second mobile component 412 and a guide rail 413, and the first mobile component 411 and the second mobile component 412 are movably arranged on the guide rail 413. The hybrid mechanism 42 includes a first connecting rod 421, a second connecting rod 422 and a bracket 423, the first end of the first connecting rod 421 is hinged to the first mobile component 411, the first end of the second connecting rod 422 is hinged to the second mobile component 412, the second end of the first connecting rod 421 and the second end of the second connecting rod 422 are respectively rotatably connected to the bracket 423, the bracket 423 is rotatably connected to the swivel mechanism 43, and the end of the swivel mechanism 43 is used to connect with an actuator for picking fruits and vegetables, and the actuator can specifically be a robot arm 50.

[0026] like Figure 6As shown, the robot arm 40 shown in this embodiment can drive the rotating mechanism 43 to move toward the target position on the guide rail 413 through the movement of the first moving component 411 and the second moving component 412 during the actual picking operation. The first moving component 411 and the second moving component 412 move toward or away from each other, so as to raise or lower the height of the rotating mechanism 43 to pick fruits and vegetables at different heights at the position. At the same time, through the rotation of the rotating mechanism 43, the robot arm 50 of the rotating mechanism 43 can be accurately rotated to the fruits and vegetables to be picked.

[0027] Compared with the prior art, the first connecting rod 421, the second connecting rod 422 and the guide rail 413 form a stable triangular structure, which makes the structure as a whole have greater rigidity; at the same time, driven by the mobile platform 41, the hybrid mechanism 42 and the rotary mechanism 43 can have a faster moving speed; in addition, only the relative distance between the first mobile component 411 and the second mobile component 412 is controlled to control the height of the rotary mechanism 43, and only the rotation angle of the rotary mechanism 43 is controlled to rotate the end manipulator 50 of the rotary mechanism 43 to the fruits and vegetables to be picked, which is relatively easy to control and can make The rotating mechanism 43 can move in a large range in the horizontal and vertical directions, and the working coverage space is larger; finally, when the relative distance between the first moving component 411 and the second moving component 412 is the farthest, the first connecting rod 421 and the second connecting rod 422 can almost be parallel to the guide rail 413, that is, the overall height is reduced, and the occupied space is smaller, making it easier for the overall structure to pass under a relatively compact space environment such as a greenhouse rack; in summary, the robotic arm 40 shown in this embodiment can simultaneously meet the requirements of high rigidity, fast movement speed, easy control, large working coverage space and small occupied space.

[0028] It should be noted that the harvesting robot provided in the embodiment of the present invention is applicable to both hanging and column-type cultivation modes. Figure 5 As shown, when picking the hanging high-rise fruits and vegetables, the robot is located at the bottom of the cultivation rack 90, and the manipulator 50 picks from both sides of the cultivation rack 90. Figure 2 and Figure 3 As shown, when picking fruits and vegetables on the column-type elevated racks, the robot is located between two rows of vertical elevated fruits and vegetables, and the manipulators 50 on both sides pick fruits and vegetables on the side of the cultivation racks 90 on both sides close to the manipulator 40. The manipulator 40 provided by the present invention has a large and flexible range of motion and can be applied to different row spacings.

[0029] In some embodiments, the hybrid mechanism 42 shown in this embodiment also includes a first gear and a second gear; the first gear is connected to the second end of the first connecting rod 421, and the second gear is connected to the second end of the second connecting rod 422; the first gear and the second gear are respectively rotatably connected to the bracket 423, and the first gear is meshed with the second gear; wherein the lengths of the first connecting rod 421 and the second connecting rod 422 are equal, and the transmission ratio between the first gear and the second gear is equal to 1.

[0030] In some embodiments, the bracket 423 shown in this embodiment has a mounting surface, the rotating mechanism 43 is rotatably connected to the mounting surface, the plane formed by the X-axis and the Y-axis is set at an acute angle to the mounting surface, and the rotating mechanism 43 is inclined toward the negative direction of the Z-axis, wherein any two of the X-axis, the Y-axis and the Z-axis are perpendicular to each other.

[0031] Specifically, by tilting the rotating mechanism 43 toward the negative direction of the Z axis, on the one hand, it can ensure that the rotating mechanism 43 manipulator 50 can obtain fruits and vegetables, and on the other hand, it makes the center of gravity of the whole body closer to the guide rail 413, that is, the height of the center of gravity of the whole body is lowered, and the overall stability is improved; at the same time, through the engagement of the first gear and the second gear, the first connecting rod 421 and the second connecting rod 422 are symmetrically opened or closed, so that the angle between the plane formed by the X-axis and the Y-axis and the installation surface can remain unchanged, so that the rotating mechanism 43 can be stably lifted and lowered along the direction of the Z axis, thereby ensuring the stability of the rotating mechanism 43 when picking fruits and vegetables.

[0032] In some embodiments, the first movable component 411 and the second movable component 412 shown in this embodiment have the same structure, and both include a slider, a base and a driving member; the slider is movably disposed on the guide rail 413, the base and the driving member are disposed on the slider, the first end of the first connecting rod 421 is hinged to the base of the corresponding first movable component 411, and the first end of the second connecting rod 422 is hinged to the base of the corresponding second movable component 412; the driving member is used to drive the slider to move along the guide rail 413.

[0033] Specifically, when the two sliders move toward each other at the same speed, the slewing mechanism 43 robot 50 can be lifted in the direction of the Z axis; when the two sliders move away from each other at the same speed, the slewing mechanism 43 robot 50 can be lowered in the direction of the Z axis; when the two sliders move at the same speed and in the same direction, the slewing mechanism 43 robot 50 can remain in the same height plane to perform picking tasks; the mobile platform 41 is loaded by the two sliders together, and the movement speed is fast. The movement of the two sliders toward or away from each other can realize the folding and unfolding of the hybrid mechanism 42 in space, achieving the effect of covering a larger space when the whole is in working state and occupying a smaller space when not in working state.

[0034] Among them, the first end of the first connecting rod 421 is connected to the corresponding base through a bearing, and the first end of the second connecting rod 422 is connected to the corresponding base through a bearing; the driving member can be a stepper servo motor, and the output shaft of the stepper servo motor is connected to a roller, and the roller is connected to the guide rail 413, and the slider is driven to move on the guide rail 413 by driving the roller to rotate.

[0035] In some embodiments, the rotary mechanism 43 shown in this embodiment includes a first rotary drive and a first rotary arm 431; one end of the first rotary arm 431 is connected to the bracket 423 through the first rotary drive. The first rotary drive drives the first rotary arm 431 to rotate relative to the bracket 423, thereby moving the manipulator 50 to the fruits and vegetables to be picked.

[0036] In some embodiments, the rotating mechanism 43 shown in this embodiment also includes a second rotating drive component and a second rotating arm 432; one end of the second rotating arm 432 is connected to the other end of the first rotating arm 431 through the second rotating drive component, and the manipulator 50 is connected to the other end of the second rotating arm 432.

[0037] Among them, the second rotary drive component can be a motor, the output shaft of the motor is connected to one end of the second rotary arm 432, and the outer casing of the motor is connected to the other end of the first rotary arm 431. When the motor is turned on, the second rotary arm 432 rotates relative to the first rotary arm 431, that is, the entire rotary mechanism 43 is a two-stage structure, the first rotary arm 431 is equivalent to the upper arm, and the second rotary arm 432 is equivalent to the lower arm. The two-stage structure increases the coverage range of the rotary mechanism 43, and at the same time, combined with the movement of the slider, the angle of the actuator is changed, so that the actuator can bypass certain obstacles and accurately reach the fruits and vegetables to be picked.

[0038] The robot arm 40 of the present embodiment connects the first connecting rod 421 and the second connecting rod 422 in parallel, and connects the first rotating arm 431 and the second rotating arm 432 in series. This combination realizes the control of the movement position and angle of the robot 50 in three-dimensional space. The entire robot arm 40 has two rotating pairs and two moving pairs, which is equivalent to a four-degree-of-freedom robot arm 40. The movements between the two rotating pairs and the two moving pairs are independent of each other. Therefore, no coupling interference occurs during the movement. This design enables the robot arm 40 to move freely in a larger working space, simplifies the control algorithm of the robot arm 40, and enhances the flexibility of the picking process.

[0039] In some embodiments, the rotation axis 52 line of the first rotating arm 431 shown in this embodiment is parallel to the rotation axis 52 line of the second rotating arm 432, that is, the output shafts of the two motors controlling the rotation are parallel, and combined with the movement of the slider, the rotation angle of the manipulator 50 can be controlled; when the first rotating arm 431 and the second rotating arm 432 are rotated and unfolded in a straight line, the actuator can extend to the farthest end, and when the first rotating arm 431 and the second rotating arm 432 are rotated to a folded state, the space occupied by the entire rotating mechanism 43 can be reduced, so that the whole can pass through a smaller space, which is convenient for free movement under the greenhouse rack.

[0040] The manipulator 50 provided in the embodiment of the present invention includes a plurality of knuckles 51 and a locking assembly 53, wherein two adjacent knuckles 51 are rotatably connected, and a locking assembly 53 is further provided between two adjacent knuckles 51; the locking assembly 53 includes a clamping member and a limiting member, and the clamping member and the limiting member are respectively arranged on the end side walls of two adjacent knuckles 51; when grasping a target object, the limiting member cooperates with the clamping member, and the knuckle 51 is unidirectionally bent toward the inner wall of the knuckle 51, and when the rigid-flexible hybrid envelope manipulator 50 puts down the target object, the limiting member is separated from the clamping member, and the knuckle 51 is reset and straightened under the action of external force.

[0041] The manipulator 50 of the present invention can freely select the number of knuckles 51 according to the application scenario, such as Figure 7 As shown, the rigid-flexible hybrid enveloping manipulator 50 includes three knuckles 51, namely the first knuckle 51, the second knuckle 51 and the third knuckle 51. The first knuckle 51 and the second knuckle 51 are rotatably connected, and the second knuckle 51 and the third knuckle 51 are rotatably connected, that is, the first knuckle 51 can rotate relative to the second knuckle 51, and the second knuckle 51 can also rotate relative to the first knuckle 51, the second knuckle 51 can rotate relative to the third knuckle 51, and the third knuckle 51 can also rotate relative to the second knuckle 51. The rigid-flexible hybrid enveloping manipulator 50 has good flexibility and adaptability.

[0042] A locking assembly 53 is provided between each pair of adjacent knuckles 51 to limit the movement of two adjacent knuckles 51. The locking assembly 53 includes a clamping member and a limiting member, wherein the limiting member is provided on the side wall of the first knuckle 51 and is arranged close to the second knuckle 51, and the clamping member is correspondingly provided on the side wall of the second knuckle 51 and is arranged close to the first knuckle 51. Similarly, a limiting member is also provided on the side wall of the second knuckle 51 and is arranged close to the third knuckle 51, and a clamping member is also provided on the third knuckle 51 and is arranged close to the second knuckle 51.

[0043] When it is necessary to grasp the target object, the limiter cooperates with the clamping part. Under the action of the driving force, the second knuckle 51 and the third knuckle 51 can bend toward the inner wall of the knuckle 51, providing a stable envelope cavity, which can generate sufficient clamping force when grasping the target object. The limiter limits the reverse bending of the knuckle 51, thereby stably grasping the object.

[0044] The inner wall surface of each knuckle 51 is also provided with a flexible contact layer 56, which can be fixedly connected to the side wall of the knuckle 51 by bolts. The flexible contact layer 56 increases the contact area between the knuckle 51 and the target object, improves the enveloping effect, and reduces damage to the target object. The knuckle 51 can be made of a rigid material, and the contact surface with the target object is a flexible contact layer 56. Combining the high precision and stable grasping of the rigid material with the strong compliance and soft grasping of the flexible contact layer 56, the grasping ability is improved.

[0045] When the target object needs to be put down, the limiter and the snap-connector are separated. Under the action of the driving force, the knuckle 51 is reset and straightened. The limiter and the snap-connector cooperate to limit the reverse bending of the knuckle 51. The rigid-flexible hybrid envelope manipulator 50 can be quickly released when putting down the object, thereby quickly and efficiently transporting the target object.

[0046] In the embodiment of the present invention, there are multiple ways to drive multiple finger joints 51 to bend. For example, a flexible cavity 54 is set between two adjacent finger joints 51, and the flexible cavity 54 is connected to an external device through a pipeline 55. Gas is simultaneously input or output into or out of the multiple flexible cavities 54 through the external device, and the pressure in the flexible cavity 54 is adjusted to enable the multiple finger joints 51 to move synchronously.

[0047] When the pressure in the flexible cavity 54 is greater than the preset pressure, the flexible cavity 54 expands and deforms, and the flexible cavity 54 squeezes the side walls of the two side knuckles 51, causing the two side knuckles 51 to bend, forming an envelope, and grasping the target object. In this process, the pressure at the connection of the knuckles 51 will be fed back to each flexible cavity 54, so that the rigid-flexible hybrid envelope manipulator 50 can adaptively adjust the bending torque according to the shape of the target object to achieve the best envelope to grasp the target object.

[0048] When the pressure in the flexible cavity 54 is less than the preset pressure, the rigid-flexible hybrid envelope manipulator 50 gradually and smoothly transitions from a bent state to an extended state.

[0049] This embodiment simultaneously adjusts the pressure in multiple flexible cavities 54 so that each finger joint 51 adaptively distributes the bending moment, thereby ensuring a simple structure while avoiding the phenomenon of concentrated pressing force caused by an under-actuated structure, thereby reducing damage to the target object.

[0050] The rigid-flexible hybrid enveloping manipulator 50 provided in the embodiment of the present invention has knuckles 51 that bend unidirectionally under the cooperation of the clamping member and the limiting member to grasp the target object. After the clamping member and the limiting member are separated, the knuckles 51 can be reset and straightened under the action of the driving force. When grasping the target object, the present invention can adaptively distribute the bending moment, limit the reverse rotation of the knuckles 51 while ensuring active enveloping grasping, provide sufficient clamping force, avoid the concentration of the clamping force and the sudden change of the clamping force caused by external factors, and reduce the damage to the target object. The limiting member in the embodiment of the present invention includes a ratchet 531, which is arranged on the side wall of the knuckle 51, and the clamping member includes a pawl 532. A plurality of ratchet 531 teeth are arranged on the outer periphery of the ratchet 531, and the first end of the pawl 532 is elastically connected to the knuckle 51, and the second end of the pawl 532 is embedded between two adjacent ratchet 531 teeth; the second end of the pawl 532 is provided with an inclined surface on the side close to the inner wall of the knuckle 51, and the inclined surface is abutted against the teeth of the ratchet 531 to provide guidance.

[0051] The locking assembly 53 in the embodiment of the present invention further includes an elastic member 534 , one end of which is connected to the side wall of the knuckle 51 , the elastic member 534 can be extended and retracted along the extension direction of the knuckle 51 , and the other end of the elastic member 534 is connected to the first end of the pawl 532 .

[0052] The elastic member 534 in the embodiment of the present invention is a spring. The side wall of the knuckle 51 is provided with a receiving groove, and the spring is received in the receiving groove. The pawl 532 is received in the receiving groove, or part of the pawl 532 extends out of the receiving groove and is embedded between two adjacent ratchet 531 teeth.

[0053] The locking assembly 53 in the embodiment of the present invention also includes a cable 533, the fixed end of the cable 533 is connected to the pawl 532, and the free end of the cable 533 extends along the extension direction of the knuckle 51; under the action of external force, the pawl 532 is driven to compress the elastic member 534 and separate from the ratchet 531; when the external force is released, the second end of the pawl 532 is embedded between the teeth of two adjacent ratchet wheels 531 under the elastic force of the elastic member 534.

[0054] In the embodiment of the present invention, two adjacent ratchet wheels 531 are rotatably connected via a rotating shaft 52, and the free end of the cable 533 is extended along the extension direction of the knuckle 51 after winding around the adjacent rotating shaft 52. The winding direction of the cable 533 is opposite to the rotation direction of the rotating shaft 52. Under the action of external force, the cable 533 drives the rotating shaft 52 to rotate in the opposite direction.

[0055] In the embodiment of the present invention, a accommodating cavity is provided between the ends of any two adjacent knuckles 51, and a flexible cavity 54 is provided in the accommodating cavity. The flexible cavity 54 is used to communicate with external equipment to adjust the pressure in the flexible cavity 54; when the pressure in the flexible cavity 54 is greater than the preset pressure, under the action of the extrusion force, the knuckles 51 bend toward the inner wall surface of the knuckles 51 for grasping the target object; when the pressure in the flexible cavity 54 is less than or equal to the preset pressure, the knuckles 51 are reset and straightened.

[0056] The flexible cavity 54 in the embodiment of the present invention includes a plurality of cavities, and two adjacent cavities are connected.

[0057] The rigid-flexible hybrid envelope manipulator 50 in the embodiment of the present invention further includes a flexible contact layer 56 , and the inner wall surface of each finger joint 51 is provided with a flexible contact layer 56 .

[0058] refer to Figure 8 Each mobile mechanism 12 includes an adapter 121, a steering assembly 122, a shock absorbing assembly 123 and a wheel assembly 124. The steering assembly 122 includes a steering seat and a steering motor. The steering seat is arranged on the frame 11 through the adapter 121, and the steering motor is configured to drive the steering seat to rotate relative to the adapter 121; the shock absorbing assembly 123 includes a cantilever, a shock absorbing damping rod and an elastic member 534. One end of the shock absorbing damping rod is hinged to the steering seat, the other end of the shock absorbing damping rod is hinged to one end of the cantilever, the other end of the cantilever is hinged to the steering seat, and the elastic member 534 is sleeved on the outer periphery of the shock absorbing damping rod; the wheel assembly 124 includes a wheel and a hub motor. The wheel is rotatably arranged on the cantilever, and the hub motor is configured to drive the wheel to rotate relative to the cantilever.

[0059] The steering seat is cast aluminum, and the steering motor can use a high-precision servo motor, which can be connected to the steering seat through a reducer, and precise angle adjustment can be achieved through programming control. The cantilever is made of high-strength carbon fiber material, which is both light and strong. The shock-absorbing damping rod uses an oil-gas shock absorber with adjustable damping, and a rubber spring is installed around the shock-absorbing damping rod to provide additional buffering effect and enhance driving stability. The wheel is an omnidirectional wheel made of high-strength polyurethane material, which has good wear resistance and grip and supports multi-directional rolling. Each wheel has an independent brushless DC motor (hub motor) built in, which directly drives the wheel to rotate, with rapid response and high power transmission efficiency. The wheel is installed on the cantilever through a bearing, and the hub motor is closely connected to the wheel shaft. The speed and steering can be precisely controlled through the controller.

[0060] In the embodiment of the present invention, while achieving all-round movement, the shock absorbing assembly 123 improves the driving stability on uneven ground, reduces the damage to precision components caused by vibration, extends the service life, and ensures that the wheels do not slide lateraly, thereby facilitating the accurate construction of the chassis kinematic model; in addition, the frame 11 and the moving mechanism 12 adopt a modular design, which makes the replacement and maintenance of each part more convenient, reduces maintenance time and cost, and improves flexibility.

[0061] In the embodiment of the present invention, a first hinge point at one end of the shock-absorbing and damping rod on the steering seat and a second hinge point at the other end of the cantilever on the steering seat are located at different heights.

[0062] In the embodiment of the present invention, in the height direction of the steering seat, the distance between the first hinge point and the adapter seat 121 is smaller than the distance between the second hinge point and the adapter seat 121 .

[0063] The adapter seat 121 of the embodiment of the present invention includes an adapter seat 121 body and a connecting portion. The steering seat is rotatably disposed on the adapter seat 121 body, and the connecting portion is configured to be snap-fitted with the frame 11 .

[0064] The frame 11 of the embodiment of the present invention is composed of a plurality of cross beams and longitudinal beams connected in an interlaced manner, and a slot is formed at the connecting portion, and the intersection of the cross beam and the longitudinal beam is engaged with the slot.

[0065] The steering assembly 122 of the embodiment of the present invention further includes a rotary shaft 52 bearing and a rotating flange. The steering motor is connected to the steering seat via the rotating flange, and the steering seat is connected to the adapter seat 121 via the rotary shaft 52 bearing.

[0066] The cantilever in the embodiment of the present invention comprises a first cantilever and a second cantilever arranged at an angle, the other end of the shock-absorbing damping rod is hinged to the first cantilever, and the second cantilever is hinged to the steering seat.

[0067] In the embodiment of the present invention, each moving mechanism 12 further includes a motor driver, which is disposed on the steering seat, and at least one of the steering motor and the hub motor is electrically connected to the motor driver.

[0068] The navigation system 20 provided in the embodiment of the present invention includes a radar and a camera. The camera accurately identifies the cultivation rack 90, and the radar obtains and establishes a map within the target area and locates the position of the cultivation rack 90 on the map.

[0069] Specifically, the radar maps the target area; the camera accurately identifies the cultivation rack; the radar can project the cultivation rack onto a two-dimensional plane for global planning and locate it in the map. Furthermore, the control system 80 plans the target route for the robot to walk. The navigation system 20 is not affected by the cultivation rack 90 and has strong adaptability to the planting environment.

[0070] The storage device 60 provided in the embodiment of the present invention includes: a storage frame 61, a mobile grabbing mechanism and a control system 80; the storage frame 61 is constructed as a multi-layer structure, each layer of the storage frame 61 is provided with a fixed tray for storing the fruit frame 62, the fixed tray at the top of the storage frame 61 is configured as a working station, and the fixed trays in the remaining storage frames 61 of each layer are configured as storage stations; the mobile grabbing mechanism includes a displacement component and a grabbing component, the displacement component is arranged on one side of the storage frame 61, and the grabbing component is arranged on the displacement component; the displacement component includes X-axis moving module, Y-axis moving module and Z-axis moving module, the X-axis moving module is slidably arranged on the Y-axis moving module, and the Y-axis moving module is slidably arranged on the Z-axis moving module; the grabbing assembly includes a bearing platform and a positioning claw clamp, the bearing platform is fixedly arranged on the shell of the Y-axis moving module, the positioning claw clamp is arranged on the X-axis moving module, the positioning claw clamp is used to cooperate with the edge of the top of the fruit frame 62 to limit, and the control system 80 is used to communicate with the mobile grabbing mechanism to control the mobile grabbing mechanism to transfer the fruit frame 62 between the storage station and the working station. The fruit frame 62 needs to have a higher load-bearing capacity and higher stability during the grabbing and transportation process to facilitate the handling of the fruit frame 62. In this embodiment, the grabbing mechanism includes the cooperation of the bearing platform and the positioning claw clamp, which can facilitate automatic grabbing and improve the load-bearing capacity, thereby realizing the stable transportation of the fruit frame 62.

[0071] In this embodiment, the mobile grasping mechanism can realize the movement of the grasping component in the three directions of X, Y, and Z, so as to realize the efficient transfer of the fruit frame 62, and through the coordinated design of the bearing platform and the positioning claw clamp and the structure of the fruit frame 62, the movement of the Z-direction moving module is utilized to realize the grasping and releasing of the fruit frame 62, thereby reducing the complexity of the control system 80. That is, only three degrees of freedom are used for the overall control, which reduces the complexity of the system, improves the stability of the system, and also reduces the requirements for motor power, simplifying the power supply and wiring design.

[0072] The storage frame 61 as the main part is a metal frame structure. The overall structure of the storage frame 61 is a multi-layer structure. A fixed pallet is laid on each layer. The fixed pallet can provide support for the fruit frame 62 and can be used to install various detection elements to realize the detection of various states on the work station. Please refer to the subsequent records for details.

[0073] After the fixed pallet is connected to the storage frame 61, the area on the top fixed pallet is configured as a work station through artificial setting. The top layer is configured as a work station to facilitate the loading of fruits and vegetables, that is, the work station is used to load fruits and vegetables. Specifically, when loading fruits and vegetables, the fruits and vegetables are loaded into the fruit frame 62 at the top through loading equipment or manual methods to achieve fruit and vegetable loading. Similarly, the fixed pallets in other structural layers except the top layer are considered to be set as storage layers, and storage stations for carrying fruit frames 62 are divided on each storage layer. The storage stations are used to store empty fruit frames 62 or fully loaded fruit frames 62.

[0074] The mobile grabbing mechanism is used to grab the fruit frame 62 (empty fruit frame or fully loaded fruit frame), and can transport the fruit frame 62 to the target position by moving the mobile assembly. The mobile grabbing mechanism cooperates with the control system 80 for control, so that the mobile grabbing mechanism can realize automatic operation under the control of the control system 80, thereby realizing the rotation between the empty fruit frame 62 on the storage station and the fully loaded fruit frame 62 (the fruit frame 62 loaded with fruits and vegetables) on the working station, and then realizing the dynamic storage of the fruit frame 62.

[0075] For example, after the fruits and vegetables are loaded, the fruit frame 62 on the top of the storage frame 61 can be grabbed by a mobile grabbing mechanism and moved to a storage station in the storage layer for storage. The fully loaded fruit frame 62 in the storage station is unloaded and transported to other processes by an external transfer tool after a certain period of time. At this time, the storage station is either replaced with an empty fruit frame or emptied directly.

[0076] In the specific configuration, the storage frame 61 is in an L-shaped structure as a whole, so that the storage frame 61 has a gap space, the mobile grabbing mechanism is arranged on one side of the gap space, and the carrying platform is located in the gap space. This method makes the overall structure more compact, can reduce the space occupied by the entire device, and the carrying platform can contact with the two sides of the storage frame 61 at the same time, so as to realize the efficient removal and storage of the fruit frame 62.

[0077] Furthermore, two parallel working stations are provided on the fixed tray of the top layer for classified storage; three storage stations are provided on the fixed tray of the storage layer, and the three storage stations are arranged according to the L-shaped structure of the storage frame 61. That is, two storage stations are arranged on one side of the long side of the storage frame 61, and one storage station is arranged on the short side of the storage frame 61. Specifically, the specific number of storage layers is not limited in this embodiment, and can be three layers, four layers, etc.

[0078] In the specific setting, the X-direction moving module, the Y-direction moving module and the Z-direction moving module are all screw modules. Specifically, the Z-direction moving module includes a column, a first slider and a first drive motor. The column is fixed to the ground or fixedly connected to the storage frame 61. The first slider is slidably arranged on the column. The first drive motor is arranged at one end of the column. The first drive motor is used to drive the screw to rotate so as to realize the first slider moving along the column in the Z direction. A limit switch 79 is also arranged on the column. The moving range of the first slider is limited by the limit switch 79. The Y-direction moving module includes a first beam, a second slider and a second drive motor. The first beam extends in the Y direction, and the first beam is fixedly arranged on the first slider. The second slider is slidably arranged on the first beam. The second drive motor is arranged at one end of the first beam. The second drive motor is used to drive the second slider to move along the Y direction. The X-direction moving module includes a second beam, a third slider and a third driving motor. The second beam extends in the X direction, and the second beam is fixed on the second slider. The third slider is slidably arranged on the second beam. The third driving motor is arranged at one end of the second beam, and the third driving motor is used to drive the third slider to move along the X direction. At least a part of the first beam is located in the gap space, the bearing platform is fixed on the first beam, and the positioning claw clamp is connected to the third slider, so that the positioning claw clamp can move in three directions (X, Y, and Z directions) to achieve the grasping and placement of the fruit frame 62. The storage device 60 in the embodiment of the present invention has a compact structure, high control accuracy and good stability.

[0079] like Figure 4 As shown, a fruit frame 62 is placed at the work station, and the fruit frame 62 includes a first area 621, a second area 622 and a third area 623. The first area 621 is used to accommodate target fruits and vegetables within a first weight range, the second area 622 is used to accommodate fruits and vegetables within a second weight range, and the third area 623 is used to accommodate fruits and vegetables within a third weight range; wherein the first weight range is greater than the second weight range, and the second weight range is greater than the third weight range; the embodiment of the present invention distinguishes fruits and vegetables of different weights by setting different areas on a fruit frame 62.

[0080] In order to facilitate the measurement of the weight of fruits and vegetables, a weight sensor is provided on the manipulator 50 in the embodiment of the present invention, and the weight sensor is used to obtain the weight of the target fruits and vegetables; the control system 80 receives the weight information obtained by the weight sensor, controls the manipulator 40 to drive the manipulator 50 to move to the fruit frame 62 within the target weight range, and places it in one of the first area 621, the second area 622 and the third area 623, thereby realizing the classification of fruits and vegetables.

[0081] The harvesting robot provided in the embodiment of the present invention further includes a pressure sensor, which is disposed on the manipulator 50 and is used to obtain the gripping force of the manipulator 50. Further, the pressure sensor is communicatively connected with the control system 80, and the control system 80 obtains the pressure information of the pressure sensor and adjusts the gripping force of the manipulator 50 based on the pressure information.

[0082] The harvesting robot in the embodiment of the present invention further includes a battery system 70, which can be installed at the bottom of the frame 11 and has a high structural strength. Fig. 9 As shown, the battery system 70 includes a mounting frame 73 and a battery module. The mounting frame 73 is provided with a mounting cavity, and the battery module is mounted in the mounting cavity. The mobile device 10, the navigation system 20, the robot arm 40, the robot hand 50, the storage device 60 and the control system 80 are electrically connected to the battery module respectively.

[0083] Specifically, the battery module includes a first battery module 71 and a second battery module 72, wherein the first battery module 71 can be a main battery module and the second battery module 72 can be a secondary battery module. The accommodating cavity includes a first mounting cavity and a second mounting cavity, which are respectively used to accommodate the first battery module 71 and the second battery module 72. The first battery module 71 and the second battery module 72 are respectively provided with a charging port 77 and an interface 76. Fig.11 The mobile device 10, the navigation system 20, the robot arm 40, the robot hand 50, the storage device 60 and the control system 80 are respectively electrically connected to the first battery module 71 or the second battery module 72 through the interface 76. The first battery module 71 and the second battery module 72 can be connected to external devices through the corresponding charging port 77 to achieve charging.

[0084] The battery system 70 of the embodiment of the present invention can realize a fast battery replacement function by providing a first battery module 71 and a second battery module 72. When one of the two is out of power, the other battery module can be used to supply power and continue picking.

[0085] In order to stably fix the battery module in the corresponding accommodating cavity, the embodiment of the present invention adopts a quick release lock 74 for fixing. The quick release lock 74 is arranged at the bottom of the mounting frame 73. The quick release lock 74 has a toggle portion 741. When the toggle portion 741 is turned upward, the toggle portion 741 can control the internal limit structure to limit the position of the battery module. When the battery module needs to be replaced, the toggle portion 741 can be pushed downward to unlock it; then hold the handle 75 on the battery module with both hands to replace the battery module. Fig. 9 and Fig.10As shown, the battery system 70 also includes a display module 78, which is used to display the current power of the battery module. The first battery module 71 and the second battery module 72 are both provided with a display module 78. The battery system 70 also includes a switch 79, and the user can control the opening and closing of the battery module through the switch 79. The first battery module 71 and the second battery module 72 are both provided with a switch. The switch 19 in the embodiment of the present invention may be a knob switch. The battery module is also provided with a communication interface, and the control system is connected to the battery module through the communication interface for controlling the opening and closing of the battery module. It should be noted that the first battery module 71 and the second battery module 72 are both provided with a communication interface, and the control system can control any battery module.

[0086] refer to Fig.12 and Fig.13 Based on the picking robot in any of the above embodiments, an embodiment of the present invention provides a picking method of a picking robot, including: step 100, identifying the position of the cultivation rack, and establishing a target area map, the target area map including the position of the cultivation rack and the driving route of the picking robot. Specifically, a radar is used to establish a global three-dimensional map of the fruit and vegetable area as the initial basis for the navigation of the picking robot. The three-dimensional map can be converted into a two-dimensional plane map to reduce the complexity of the operation, reduce the calculation time, and improve the practicality of navigation. The position of the cultivation rack is obtained by using a camera, and the cultivation rack is accurately identified by an image recognition algorithm, and the position of the cultivation rack is displayed on the target area map. Based on the position of the cultivation rack on the target area map, a driving route of the picking robot can be established, such as a Z-shaped route and a bow-shaped route.

[0087] Step 200, based on the initial position of the picking robot and the position of the target cultivation rack, determine the target route; specifically including: identifying the first obstacle through radar and performing a first positioning; the first obstacle includes a wall; identifying the cultivation rack through a visual system and performing a second positioning; fitting the target route based on the first positioning and the second positioning. The embodiment of the present invention identifies the first obstacle in the target area, such as a wall, through radar; performs coarse positioning, i.e., the first positioning; identifies the position of the cultivation rack through a visual system for precise positioning, i.e., the second positioning; further, the radar and the recognition system are dual-positioned to achieve precise positioning of the local route, and then the target route of the initial position and the target cultivation rack can be fitted.

[0088] Step 300, controlling the moving device to move to the position of the target cultivation rack according to the target route, that is, below the cultivation rack, or between two rows of cultivation racks.

[0089] Step 400, obtain the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, determine the picking order, and the picking robot can pick according to the picking order. Specifically, obtaining the complete image of the target fruits and vegetables includes: obtaining the RGB image and X-ray image of the target fruits and vegetables that are synchronized in space; fusing the RGB image and X-ray image of the target fruits and vegetables that are synchronized in space to obtain the fused image of the target fruits and vegetables; inputting the fused image of the target fruits and vegetables into the recognition model to obtain the complete image of the target fruits and vegetables output by the recognition model; wherein the recognition model is obtained by training the fused image of the target fruits and vegetables as a sample and the complete image of the target fruits and vegetables corresponding to the fused image of the target fruits and vegetables as a label; obtaining the three-dimensional position information of the target fruits and vegetables includes: judging whether the target fruits and vegetables are fully blocked by the fused image of the RGB image and the X-ray image; if not blocked or partially blocked, then aligning the depth image of the target fruits and vegetables with the synchronized RGB image of the target fruits and vegetables to determine the three-dimensional position information of the target fruits and vegetables; based on the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, the target fruits and vegetables are automatically picked.

[0090] Specifically, feature points corresponding to the RGB image and X-ray image of the target fruits and vegetables are obtained; based on the feature points corresponding to the RGB image and X-ray image of the target fruits and vegetables, the homography matrix of the RGB image and X-ray image of the target fruits and vegetables is determined; based on the homography matrix, the RGB image and X-ray image of the target fruits and vegetables are registered to obtain spatially synchronized RGB image and X-ray image of the target fruits and vegetables.

[0091] The controller can fuse the synchronized RGB image and X-ray image of the target fruits and vegetables. Since the X-ray image is a grayscale image, the controller can first fuse the RGB image and the X-ray image in grayscale, and then fuse the RGB image and the X-ray image in color to obtain a fused image of the target fruits and vegetables.

[0092] The controller inputs the fused image of the target fruits and vegetables into the recognition model, and can obtain the complete image of the target fruits and vegetables output by the recognition model.

[0093] The fused image of the target fruit and vegetable can be used as a sample to create a data set, and the complete image of the target fruit and vegetable corresponding to the fused image of the target fruit and vegetable can be annotated and used as a label to train the initial model. When the initial model meets the preset conditions, the recognition model can be obtained. The preset conditions can be a limit on the number of training times or a limit on the accuracy, which is not specifically limited in this application.

[0094] Through the fusion image of RGB image and X-ray image, the occlusion of target fruits and vegetables can be determined. When the target fruits and vegetables are not occluded or are partially occluded, the controller can obtain the depth image of the target fruits and vegetables, and use the distance information of the depth image to perform three-dimensional positioning of the target fruits and vegetables. There is a certain correspondence between the RGB image and the depth image. The controller can register the depth image of the target fruits and vegetables with the synchronized RGB image of the target fruits and vegetables, align the positions of the targets in the two images, and thus determine the three-dimensional position information of the target fruits and vegetables. When there is no occlusion of the target fruits and vegetables, the three-dimensional position information of the target fruits and vegetables can be directly determined based on the registered and aligned images; when there is partial occlusion of the target fruits and vegetables, the three-dimensional position information of the target fruits and vegetables can be determined by the depth information of the registered and aligned images of the unoccluded part of the target fruits and vegetables. Specifically, the 2D bounding box of the identified target fruits and vegetables can be generated into a 3D bounding box based on the depth information contained in the depth image of the target fruits and vegetables to obtain the three-dimensional position information of the target fruits and vegetables, thereby performing three-dimensional positioning of the target fruits and vegetables. Based on the complete image of the identified target fruits and vegetables and the acquired three-dimensional position information of the target fruits and vegetables (such as Figure 6 As shown), the target fruits and vegetables can be automatically picked.

[0095] The positioning and identification method for automatic picking of fruits and vegetables in an occluded environment provided in the embodiment of the present application determines the three-dimensional position of the target fruits and vegetables by aligning the depth image of the target fruits and vegetables with the synchronized RGB image of the target fruits and vegetables, thereby automatically picking the target fruits and vegetables. This can achieve rapid and accurate positioning of the target fruits and vegetables and ensure the smooth execution of the automatic picking steps.

[0096] Step 500, based on the picking order, corresponding to the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, the robotic arm is controlled to drive the robotic arm to move toward the target fruits and vegetables, the robotic arm is located at the picking position of the target fruits and vegetables, and the robotic arm is controlled to pick the target fruits and vegetables.

[0097] Step 600, obtaining the current weight of the target fruits and vegetables. Specifically, based on a weight sensor provided on the manipulator, the current weight of the target fruits and vegetables is obtained.

[0098] Further, in step 700, based on the current weight, the robot is controlled to place the target fruit in the target fruit frame of the storage device. Specifically, the current weight is compared with the target weight range in the target area of ​​each fruit frame, and based on the comparison result, the robot is controlled to place the target fruit and vegetable in the target area of ​​the storage device.

[0099] The picking robot and picking method provided by the embodiments of the present invention can be applied to fruits and vegetables such as strawberries, apples, and tomatoes.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A picking robot, It is characterized in that include: The mobile device comprises: a frame and a plurality of mobile mechanisms, wherein the plurality of mobile mechanisms are arranged on the frame; A navigation system, provided on the rack, for identifying the location of the cultivation rack and establishing a map of the target area; A visual system, arranged on the frame, for obtaining a complete image of target fruits and vegetables and three-dimensional position information of the target fruits and vegetables; A mechanical arm is arranged on the frame, and a mechanical hand is arranged at the free end of the mechanical arm for picking target fruits and vegetables; A storage device, provided on the frame, for storing the target fruits and vegetables picked by the robot; A control system, wherein the mobile device, the navigation system, the visual system, the robotic arm, the robotic hand, and the storage device are respectively connected to the control system for communication; The control system determines the target route based on the target position and the target area map; controls the movement of the mobile device based on the target route; determines the picking order based on the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, controls the robot arm and the robot hand to move to pick the target fruits and vegetables, and places them in the storage device.

2. The picking robot according to claim 1, It is characterized in that The navigation system includes a radar and a camera. The camera accurately identifies the cultivation rack. The radar acquires and establishes a target area map and locates the position of the cultivation rack on the map.

3. The picking robot according to claim 1, It is characterized in that The storage device comprises a storage frame and a mobile grabbing mechanism, wherein the storage frame is constructed as a multi-layer structure, each layer of the storage frame is provided with a fixed pallet for storing fruit boxes, the fixed pallet at the top of the storage frame is configured as a working station, and the fixed pallets in the storage frames of the remaining layers are configured as storage stations; The mobile grasping mechanism includes a displacement component and a grasping component, the displacement component is arranged on one side of the storage frame, and the grasping component is arranged on the displacement component; the displacement component includes an X-direction moving module, a Y-direction moving module and a Z-direction moving module, the X-direction moving module is slidably arranged on the Y-direction moving module, and the Y-direction moving module is slidably arranged on the Z-direction moving module; the grasping component includes a bearing platform and a positioning claw clamp, the bearing platform is fixedly arranged on the shell of the Y-direction moving module, the positioning claw clamp is arranged on the X-direction moving module, and the positioning claw clamp is used to cooperate with a part of the edge of the top of the fruit frame to limit the position; the mobile grasping structure is communicatively connected with the control system, and is used to control the mobile grasping mechanism to transfer the fruit frame between the storage station and the working station.

4. The picking robot according to claim 3, It is characterized in that It also includes a weight sensor, which is arranged on the manipulator and is used to obtain the weight of the target fruits and vegetables; The fruit frame includes a first area, a second area and a third area, the first area is used to accommodate target fruits and vegetables within a first weight range, the second area is used to accommodate fruits and vegetables within a second weight range, and the third area is used to accommodate fruits and vegetables within a third weight range; wherein the first weight range is greater than the second weight range, and the second weight range is greater than the third weight range; The weight sensor is in communication with the control system and is used to control the robot to place the target fruits and vegetables in one of the first area, the second area and the third area.

5. The picking robot according to claim 1, It is characterized in that It also includes a pressure sensor, which is arranged on the manipulator and is used to obtain the gripping force of the manipulator; the pressure sensor is communicatively connected with the control system.

6. The picking robot according to claim 1, It is characterized in that Also includes: A battery system, the battery system comprising a mounting frame and a battery module, the mounting frame being mounted on the frame, the mounting frame having a mounting cavity, the battery module being mounted in the mounting cavity, the mobile device, the navigation system, the robotic arm, the robotic hand, the storage device and the control system being electrically connected to the battery module respectively.

7. The picking robot according to claim 6, It is characterized in that The battery module includes a first battery module and a second battery module, and the installation cavity includes a first installation cavity and a second installation cavity, which are respectively used to accommodate the first battery module and the second battery module, and the first battery module and the second battery module are respectively provided with interfaces; the mobile device, the navigation system, the robotic arm, the robotic hand, the storage device and the control system are respectively electrically connected to the first battery module or the second battery module through the interface.

8. A picking method using a picking robot, based on the picking robot according to any one of claims 1 to 7, It is characterized in that include: Identify the location of the cultivation rack and establish a target area map, wherein the target area map includes the location of the cultivation rack and the driving route of the picking robot; Determine the target route based on the initial position of the picking robot and the position of the target cultivation rack; Controlling the mobile device to move to the position of the target cultivation rack according to the target route; Obtain the complete image and three-dimensional position information of the target fruits and vegetables, and determine the picking order; Based on the picking order, corresponding to the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, the robotic arm is controlled to drive the robotic arm to move toward the target fruits and vegetables, and the robotic arm is controlled to pick the target fruits and vegetables; Obtaining the current weight of the target fruit and vegetable; Based on the current weight, the robot is controlled to place the target fruit in a target area of ​​the storage device.

9. The picking method of the picking robot according to claim 8, It is characterized in that Based on the initial position of the picking robot and the position of the target cultivation rack, determine the target route; including: Identifying a first obstacle by radar and performing a first positioning; the first obstacle includes a wall; The cultivation rack is identified by the visual system and positioned for the second time; Based on the first positioning and the second positioning, a target route is fitted.

10. The picking method of the picking robot according to claim 8, It is characterized in that Obtain complete images and 3D location information of target fruits and vegetables, including: Acquire spatially synchronized RGB images and X-ray images of target fruits and vegetables; The spatially synchronized RGB image and X-ray image of the target fruits and vegetables are fused to obtain a fused image of the target fruits and vegetables; Inputting the fused image of the target fruits and vegetables into the recognition model to obtain the complete image of the target fruits and vegetables output by the recognition model; The recognition model is obtained by training by taking the fused image of the target fruits and vegetables as a sample and the complete image of the target fruits and vegetables corresponding to the fused image of the target fruits and vegetables as a label; The fusion image of RGB image and X-ray image is used to determine whether the target fruits and vegetables are completely blocked; If it is not blocked or partially blocked, the depth image of the target fruit and vegetable is registered with the synchronized RGB image of the target fruit and vegetable to determine the three-dimensional position information of the target fruit and vegetable; Based on the complete image of the target fruits and vegetables and the three-dimensional position information of the target fruits and vegetables, the target fruits and vegetables are automatically picked.

Images