Robot strawberry picking anthropomorphic operation data collection equipment

Abstract

The utility model discloses robot strawberry picking anthropomorphic operation data collection equipment which comprises a handheld grip arranged below a mounting platform; the picking actuator structure is arranged at the front end of the upper part of the mounting platform and comprises a pair of picking fingers extending out of the mounting platform, each picking finger is in sliding connection with the mounting platform through a mounting seat, two groups of sliding rails are arranged on the mounting platform, and sliding blocks matched with one group of sliding rails are respectively arranged on the two mounting seats; the two mounting seats are respectively provided with an operating rod which downwards extends out of the mounting platform; each mounting seat is also provided with an actuator opening degree positioning auxiliary label; the binocular camera is arranged on the mounting platform, and the picking finger and the actuator opening degree positioning auxiliary label can be seen through the view field angle of the binocular camera; when the picking device is used, an operator holds the handheld grip by hand and controls the two picking fingers to move through the operating rod, and then the binocular camera collects space attitude information of the picking fingers. The collection device can greatly improve the collection efficiency of the robot anthropomorphic data.

Description

Technical Field

[0001] The utility model relates to the technical field of artificial intelligence, in particular to a data collection device for anthropomorphic operation of robot strawberry picking. Background Technique

[0002] At present, there is no commercial strawberry harvesting robot system. An important reason is that agricultural robots face the operation of deformable objects in a complex unstructured biological environment when picking strawberries. It is very difficult to operate safely in such a space full of rigid-flexible hybrid obstacles and without precise operation space. Not only the target fruit needs to be sensed, but also the entire complex scene needs to be sensed, fully understanding the leaves, steam, fruits, obstacles and their spatial relationships. The physical properties of all objects also need to finally calculate a strategy to avoid obstacles, push aside messy soft objects, and reach the precise picking point above the calyx directly above the target fruit stem. This poses huge challenges in all aspects of the robot, including scene understanding, depth estimation, trajectory planning and control, which are at or beyond the level of the current state-of-the-art robot systems. However, the recently emerging new generation of imitation learning methods, including Action Chunk Transformer (ACT) and Diffusion Policy method (DP), have revolutionarily solved the long-term problem of robot manipulation, which requires fine-grained visual closed-loop interaction with non-standard non-rigid objects. Previously very difficult robot skills, such as folding clothes, threading wires, pouring sauce and spreading on pizza dough, can now be mastered by the robot through learning human teaching operations, which also provides new possibilities for the robot to learn strawberry picking by imitating human operations.

[0003] However, in this kind of imitation learning, the acquisition of human teaching data is crucial. Currently, the industry and academia generally collect data through twin robotic arms. Such a data collection device has the same size and degrees of freedom as the robotic arm, and the data collection of robot operations is directly remotely operated by a person on-site. The advantage is that this system can completely replicate robot operations and obtain the angle information of each joint for the subsequent imitation learning process. However, such a system is limited by cost, size and data collection convenience, and can only collect data in a small range and on a small scale. Moreover, it is very difficult for such a system to be used in a narrow, cramped and complex agricultural scene outside the laboratory space.

[0004] Another type of data collection method is teleoperation through VR devices. The robot instructor wears a VR headset, and the VR device identifies the positions and postures of the human arm and fingers (or VR handle) and maps them to the actuator postures of the robot to collect robot operation data. This type of teleoperation data collection system has the advantages of small size and commercialization, but it also has the disadvantages of lack of force feedback, a gap between the operator's spatial perception and the actual situation, and only being suitable for humanoid robots. Moreover, the price of VR devices with sufficient accuracy is still very high at present (generally Apple Vision Pro).

[0005] Therefore, there is currently a need for a set of collection devices that can achieve low cost, portability, and can collect artificial operation data that can be provided for robot learning on a large scale in agricultural scenarios. Summary of the Utility Model

[0006] Based on the deficiencies in the prior art, the purpose of the present utility model is to provide a robot strawberry picking anthropomorphic operation data collection device, which can collect available data for robots to imitate and learn strawberry picking through the actual work of artificial strawberry picking at low cost and on a large scale, so as to drive the training of the robot embodied intelligence algorithm.

[0007] Based on the above purpose, the present utility model provides a robot strawberry picking anthropomorphic operation data collection device, including: an installation platform; a handheld grip, provided below the installation platform;

[0008] A picking actuator structure, provided at the front end above the installation platform, includes a pair of picking fingers extending out of the installation platform. Each picking finger is slidably connected to the installation platform through a mounting seat. Two sets of sliding rails are provided on the installation platform, and sliders cooperating with one of the sets of sliding rails are respectively provided on the two mounting seats; an operating rod extending downward through the installation platform is provided on each of the two mounting seats; an actuator opening positioning auxiliary label is also provided on each mounting seat;

[0009] A binocular camera, provided on the installation platform. The field of view angle of the binocular camera can see the picking fingers and the actuator opening positioning auxiliary label; an inertial measurement unit (IMU) is built into the binocular camera for collecting inertial measurement unit data;

[0010] During use, the operator holds the handheld grip and controls the movement of the two picking fingers through the operating rod, and then the binocular camera collects the opening and spatial posture information of the picking fingers.

[0011] Preferably, the binocular camera is provided at the rear end above the installation platform. This can ensure that the camera has a better field of view to capture the dynamic changes of the picking fingers and the actuator opening positioning auxiliary label. This layout helps to reduce the field of view blind area and improve the accuracy and integrity of data collection.

[0012] Furthermore, the binocular camera is mounted on the mounting platform through a binocular camera mounting seat, and the binocular camera mounting seat includes a horizontal mounting plate and a vertical mounting plate;

[0013] The mounting platform is provided with first fixing screw holes, the horizontal mounting plate is provided with first oblong holes extending in the front-rear direction, and the binocular camera mounting seat is fixed by first bolts passing through the first oblong holes and mating with the first fixing screw holes;

[0014] The back of the binocular camera is provided with second fixing screw holes, the vertical mounting plate is provided with second oblong holes extending in the vertical direction, and the binocular camera mounting seat is fixed by second bolts passing through the second oblong holes and mating with the second fixing screw holes.

[0015] The above settings provide greater flexibility and adjustability. By the cooperation of the oblong holes and bolts, the position of the binocular camera can be conveniently fixed and adjusted, while ensuring the firmness of the installation. This design provides both flexibility and ensures the stability during the acquisition process.

[0016] Preferably, each group of the slide rails is one, and two of the slide rails are linearly arranged in a row, and the two slide rails are connected together or have a gap therebetween;

[0017] The mounting platform is further provided with a stop block, which is located between the two mounting seats and is used to limit the two mounting seats in the direction of approaching each other, restricting the picking fingers from approaching too closely and preventing potential damage or interference.

[0018] Preferably, the two operating rods are located in the upper front of the hand-held grip. When in use, the operator holds the hand-held grip with one hand and operates the two operating rods to move, improving the convenience and efficiency of the operation and making the data acquisition process smoother.

[0019] Preferably, in the present utility model, the mounting platform is further provided with a reset rubber band or a reset spring for automatically resetting the two operating rods to a position away from each other, reducing manual intervention and improving the continuity and efficiency of data acquisition.

[0020] Preferably, each of the mounting seats has a label setting plate protruding upward, and an actuator opening positioning auxiliary label is provided on one side of the label setting plate facing the binocular camera.

[0021] The beneficial effects of the present utility model:

[0022] Using the anthropomorphic operation data collection device for robot strawberry picking of the present utility model can greatly improve the collection efficiency of anthropomorphic data of the robot. Compared with the twin arms, the data collection efficiency of a single person is increased by more than ten times, and the collection design can be replicated at low cost and distributed to multiple people for crowdsourcing data collection without the need for a robot entity. The real-time performance of the device also meets the requirements for real-time synchronous remote control of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a front view schematic diagram of the anthropomorphic operation data collection device for robot strawberry picking of the present utility model;

[0024] Figure 2 It is a top view schematic diagram of the anthropomorphic operation data collection device for robot strawberry picking of the present utility model from above;

[0025] Figure 3 It is a bottom view schematic diagram of the anthropomorphic operation data collection device for robot strawberry picking of the present utility model from below;

[0026] Figure 4 It is a rear view schematic diagram of the anthropomorphic operation data collection device for robot strawberry picking of the present utility model;

[0027] Figure 5 It is a schematic diagram of the picking actuator structure of the present utility model;

[0028] Figure 6 It is a left view schematic diagram of the picking actuator structure of the present utility model;

[0029] Figure 7 It is a schematic diagram of the binocular camera mounting seat.

[0030] Reference numerals in the figures: 1 - mounting platform, 11 - first fixing screw hole, 12 - stop block, 2 - hand grip, 3 - picking actuator structure, 31 - picking finger, 32 - mounting seat, 321 - label setting plate, 33 - slide rail, 34 - slider, 35 - operating rod, 36 - actuator opening positioning auxiliary label, 4 - binocular camera, 41 - binocular camera mounting seat, 411 - horizontal mounting plate, 4111 - first waist-shaped hole, 412 - vertical mounting plate, 42 - second fixing screw hole. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] By Figures 1-7As shown, the utility model provides a robot strawberry picking anthropomorphic operation data collection device, comprising: a mounting platform 1; a handheld handle 2, arranged below the mounting platform 1; a picking actuator structure 3, arranged at the upper front end of the mounting platform 1, comprising a pair of picking fingers 31 extending out of the mounting platform 1, each picking finger 31 being slidably connected to the mounting platform 1 through a mounting seat 32, the mounting platform 1 being provided with two sets of slide rails 33, and the two mounting seats 32 being respectively provided with a slider 34 cooperating with one set of slide rails 33; each of the two mounting seats 32 is provided with an operating rod 35 extending downward through the mounting platform 1; each mounting seat 32 is also provided with an actuator opening positioning auxiliary label 36; a binocular camera 4, arranged on the mounting platform 1, the field of view angle of the binocular camera 4 being able to see the picking fingers 31 and the actuator opening positioning auxiliary label 36; the binocular camera 4 has a built-in IMU for collecting IMU data;

[0032] During use, the operator holds the handheld handle 2 and controls the movement of the two picking fingers 31 through the operating rod 35 , and then the binocular camera 4 collects the opening and spatial posture information of the picking fingers 31 .

[0033] The device is equipped with spatial attitude sensors: 1. A binocular camera; 2. An IMU. The IMU is built into the binocular camera and collects primary data, including binocular video and IMU data. After post-processing, secondary information is obtained, including: 1. Aperture information; 2. Spatial attitude information.

[0034] When manually operating data collection, the picking fingers are operated by operating levers. The picking fingers facilitate the actuator to push away obstructions and reach into the gap to clamp the strawberries and cut the roots and stems short through a blade on one side (a blade is set above either of the two picking fingers) (while keeping the clamped strawberries from falling).

[0035] In this utility model, the spatial attitude positioning binocular camera mounting structure mounts the binocular camera at an angle that allows it to see the actuator and the actuator opening positioning auxiliary label, while blocking all other parts. This ensures that the video data collected by the system is identical to the perspective of the robot actually operating with the electric end effector, making it impossible to distinguish from images. Two positioning labels are attached to the actuator opening positioning auxiliary label, and the distance between the labels is calculated through algorithmic processing to obtain the actuator opening information. This eliminates the need for an opening sensor on the unpowered end effector, further reducing costs and eliminating the need for complex circuit power supply.

[0036] In the present invention, in order to ensure that the camera has a better field of view to capture the dynamic changes of the picking finger and the actuator opening positioning auxiliary label, the binocular camera 4 is set at the upper rear end of the mounting platform 1 ( Figures 1-4As an example). The binocular camera 4 can be installed at the upper left rear end, the direct rear end, or the upper right rear end of the mounting platform 1, as long as it can reduce the visual blind area and improve the accuracy and integrity of data collection.

[0037] Furthermore, to provide greater flexibility and adjustability, the binocular camera 4 is mounted on the mounting platform 1 through a binocular camera mount 41. The binocular camera mount 41 includes a horizontal mounting plate 411 and a vertical mounting plate 412.

[0038] The mounting platform 1 is provided with a first fixing screw hole 11, and the horizontal mounting plate 411 is provided with a first waist-shaped hole 4111 extending in the front-rear direction. The binocular camera mount 41 is fixed by a first bolt passing through the first waist-shaped hole 4111 and mating with the first fixing screw hole 11. The back of the binocular camera 4 is provided with a second fixing screw hole, and the vertical mounting plate 412 is provided with a second waist-shaped hole 4121 extending in the vertical direction. The binocular camera mount 41 is fixed by a second bolt passing through the second waist-shaped hole 4121 and mating with the second fixing screw hole. Through the first waist-shaped hole and the second waist-shaped hole, the position and angle of the camera can be conveniently adjusted to meet different collection requirements. In addition, the cooperation between the waist-shaped hole and the bolt can also conveniently fix and adjust the position of the binocular camera while ensuring the firmness of the installation.

[0039] In the present utility model, each group of slide rails 33 is one, and the two slide rails 33 are linearly arranged in a row, and the two slide rails 33 are connected together or have a gap therebetween, which helps the picking fingers to move smoothly in a fixed direction. To prevent the picking fingers from being damaged or interfered due to excessive proximity, a stop block 12 is further provided on the mounting platform 1. The stop block 12 is located between the two mounting seats 32 and is used to limit the two mounting seats 32 in the direction of approaching each other, improving the safety and stability of the device.

[0040] In the present utility model, the two operating rods 35 are located at the upper front of the hand-held grip 2. During use, the operator can easily hold the hand-held grip 2 with one hand and operate the two operating rods 2 to move, thereby controlling the movement of the two picking fingers. This design improves the convenience and efficiency of operation, making the data collection process smoother.

[0041] To reduce manual intervention, the present utility model is also provided with a reset rubber band or a reset spring on the mounting platform 1 for automatically resetting the two operating rods 2 to the position where they are away from each other, or other tools capable of automatically resetting the two operating rods 2 to the position where they are away from each other, such as the cooperation of an electromagnet and a ferromagnetic material, as long as it can meet the requirements of improving the continuity and efficiency of data collection.

[0042] Each mounting bracket 32 has an upwardly extending label placement plate 321. A label placement plate 321, facing the binocular camera 4, is equipped with an actuator opening auxiliary label 36. The label placement plate accurately records the opening of the picking finger. The upward extension ensures that the label and the picking finger are always in the same field of view, ensuring accurate data collection.

[0043] During use, when the operator directly sees strawberries, they use one handheld handle 2 and control the movement of two picking fingers 31 via operating lever 35, clamping the strawberries and shortening the stems with blades on one side. When the operator sees strawberries hidden by obstructions (such as strawberry leaves), they use operating lever 35 to operate the picking fingers 31 to push aside the obstructions and reach into the gaps to clamp the strawberries and shorten the stems with blades on one side. In both cases, the binocular camera 4 captures the field of view angle video of the picking fingers 31 and the actuator opening positioning auxiliary label 36 on the label setting plate 321, which is the anthropomorphic strawberry picking operation data.

Claims

1. A robot strawberry picking anthropomorphic operation data collection device, characterized in that, Comprising: An installation platform; A handheld grip, provided below the installation platform; A picking actuator structure, provided at the front end above the installation platform, including a pair of picking fingers extending out of the installation platform. Each picking finger is slidably connected to the installation platform through a mounting seat. Two sets of slide rails are provided on the installation platform, and sliders cooperating with one of the sets of slide rails are respectively provided on two mounting seats; An operating rod extending downward through the installation platform is provided on each of the two mounting seats; An actuator opening positioning auxiliary label is also provided on each mounting seat; A binocular camera, provided on the installation platform. The field of view angle of the binocular camera can see the picking fingers and the actuator opening positioning auxiliary label; An inertial measurement unit is built in the binocular camera for collecting inertial measurement unit data; During use, the operator holds the handheld grip and controls the movement of the two picking fingers through the operating rod, and then the binocular camera collects the opening and spatial attitude information of the picking fingers.

2. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 1, wherein, The binocular camera is provided at the rear end above the installation platform.

3. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 2, characterized in that, The binocular camera is mounted on the installation platform through a binocular camera mounting seat. The binocular camera mounting seat includes a horizontal mounting plate and a vertical mounting plate; A first fixing screw hole is provided on the installation platform, a first slotted hole extending in the front-rear direction is provided on the horizontal mounting plate, and the binocular camera mounting seat is fixed by a first bolt passing through the first slotted hole and cooperating with the first fixing screw hole; A second fixing screw hole is provided on the back of the binocular camera, a second slotted hole extending in the vertical direction is provided on the vertical mounting plate, and the binocular camera mounting seat is fixed by a second bolt passing through the second slotted hole and cooperating with the second fixing screw hole.

4. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 1, wherein, Each set of the slide rails is one, and the two slide rails are linearly arranged in a row, and the two slide rails are connected together or there is a gap between them; A stop block is further provided on the installation platform. The stop block is located between the two mounting seats and is used for limiting the two mounting seats in the direction of approaching each other.

5. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 1, characterized in that, The two operating rods are located in front of and above the handheld grip. During use, the operator holds the handheld grip with one hand and operates the two operating rods to move.

6. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 5, characterized in that, A reset rubber band or a reset spring for automatically resetting the two operating rods to a position away from each other is further provided on the installation platform.

7. The anthropomorphic operation data collection device for robotic strawberry picking according to claim 1, wherein Each mounting seat has a label setting plate extending upward. The actuator opening positioning auxiliary label is provided on the side of the label setting plate facing the binocular camera.

Images