Robot strawberry picking anthropomorphic operation data collection equipment and data processing method thereof

Abstract

The invention discloses robot strawberry picking anthropomorphic operation data collection equipment and a data processing method thereof, and relates to the technical field of artificial intelligence. By adopting the robot strawberry picking anthropomorphic operation data collection equipment and the data processing method thereof, the acquisition efficiency of robot anthropomorphic data can be greatly improved, and compared with a twin arm, the efficiency of single-person data acquisition is improved by more than ten times; and the acquisition design can be copied at low cost, and multi-person crowdsourcing acquisition data can be distributed without a robot entity. The real-time performance of the equipment also meets the real-time synchronous remote control of the robot. In general, the anthropomorphic operation data collection device and the data processing method thereof have important efficiency improvement significance for constructing large-scale agricultural strawberry picking robot training data and obtaining an intelligent robot with a body.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data collection device for anthropomorphic operation of robot strawberry picking and a data processing method thereof. Background Art

[0002] Although many researchers have studied the robotic harvesting of strawberries since 1998 and designed a variety of robots that can automatically pick strawberries according to programs, there is currently no commercial strawberry harvesting robot system. One of the important reasons is that agricultural robots face the challenge of operating deformable objects in a complex unstructured biological environment when picking strawberries. It is very difficult to operate safely in such a space full of mixed rigid and soft obstacles and inaccurate operation. Not only the target fruit must be sensed, but also the entire complex scene must be sensed, and the physical properties of leaves, steam, fruit, obstacles and their spatial relationships must be fully understood. Finally, a strategy must be calculated to avoid obstacles, push away cluttered soft objects, and reach the precise picking point on the stem of the target fruit just above the calyx. This poses huge challenges in all aspects of robotics, including scene understanding, depth estimation, trajectory planning and control, which are at or beyond the level of current state-of-the-art robotic systems. However, the recent emergence of a new generation of imitation learning methods, including the Action Block Transformer (ACT) and the Diffusion Policy method (DP), has revolutionized the long-standing problem of robot manipulation, which requires fine-grained visual closed-loop interaction with non-standard non-rigid objects. Previously very difficult robot skills, such as folding clothes, threading harnesses, pouring sauce, and spreading pizza dough, can now be mastered by robots by learning human-taught operations, which also provides new possibilities for robots to learn to pick strawberries by imitating human operations.

[0003] However, in this type of imitation learning, the acquisition of human teaching data is crucial. The current industry and academia generally collect data through twin robotic arms. Such data collection devices have the same size and degrees of freedom as the robotic arms. People directly perform remote operations on site to collect data on robot operations. The advantage is that this system can completely replicate the robot operation and obtain the angle information of each joint to facilitate the subsequent imitation learning process. However, such a system is limited by cost, reference and convenience of data collection, and can only perform small-scale data collection. Moreover, such a system is difficult to move out of the laboratory space and use in small, cramped and complex agricultural scenarios.

[0004] Another data collection method is to use VR equipment for remote operation. The robot instructor wears a VR headset and uses the VR equipment to identify the posture of the human arm and finger (or VR handle) and map it to the robot's actuator posture to collect robot operation data. This type of remote operation data collection system has the advantages of small size and commercialization, but it also has the disadvantages of lack of force feedback, the gap between the operator's spatial perception and the actual existence, and only being suitable for humanoid robots. In addition, the price of VR equipment with sufficient accuracy is still very high (usually Apple Vision Pro).

[0005] Therefore, there is a need for a low-cost, lightweight and large-scale collection device for manual operation data in agricultural scenarios that can be provided to robots for learning. The goal of this device is to be independent of a specific robot and to be distributed in large quantities to strawberry pickers. A large number of strawberry pickers actually pick strawberries through this device. The picking operation data is recorded in this process and restored to the three-dimensional robot arm actuator end trajectory data and actuator opening data through a series of data processing methods, so that such data can be used as usable data for robots to imitate and learn strawberry picking. This device can build a robot strawberry picking action dataset that is much larger than the current imitation learning dataset, so that the robot can learn strawberry picking operations through data-driven. Summary of the invention

[0006] The purpose of the present invention is to construct a set of anthropomorphic imitation learning data acquisition devices for training robots to perform strawberry picking operations through innovative hardware and algorithm solutions. The goal of the device and the corresponding algorithm is to be able to collect and process a large amount of three-dimensional robotic arm actuator end trajectory data and actuator opening data through actual manual strawberry picking work at low cost and on a large scale as usable data for the robot to imitate and learn strawberry picking, thereby driving the robot's embodied intelligent algorithm training.

[0007] Based on the above purpose, the present invention provides a robot strawberry picking anthropomorphic operation data collection device, comprising: a mounting platform; a handheld handle, arranged below the mounting platform;

[0008] The picking actuator structure is arranged at the upper front end of the mounting platform, and includes a pair of picking fingers extending out of the mounting platform, each picking finger is slidably connected to the mounting platform through a mounting seat, and two sets of slide rails are arranged on the mounting platform, and two mounting seats are respectively provided with a slider that cooperates with one set of slide rails; each of the two mounting seats is provided with an operating rod extending downward through the mounting platform; each mounting seat is also provided with an actuator opening positioning auxiliary label;

[0009] A binocular camera is arranged on the mounting platform, and the field of view of the binocular camera can see the picking finger and the actuator opening positioning auxiliary label; the binocular camera has a built-in inertial measurement unit (IMU) for collecting inertial measurement unit data;

[0010] When in use, the operator holds the hand-held handle and controls the movement of two picking fingers through the operating rod, and then the binocular camera collects the opening and spatial posture information of the picking fingers.

[0011] This device is equipped with spatial attitude sensors: 1. Binocular camera; 2. IMU. IMU is built into the binocular camera, so the primary data collected is binocular video + IMU data. After processing, the secondary information obtained is: 1. Opening information; 2. Spatial posture information.

[0012] Preferably, the binocular camera is arranged at the upper rear end of the mounting platform. This ensures that the camera has a better field of view to capture the dynamic changes of the picking fingers and the actuator opening positioning auxiliary labels. This layout helps to reduce the blind area of ​​the field of view and improve the accuracy and completeness of data collection.

[0013] Further, the binocular camera is installed on the mounting platform through a binocular camera mounting seat, and the binocular camera mounting seat includes a horizontal mounting plate and a vertical mounting plate;

[0014] The mounting platform is provided with a first fixing screw hole, the horizontal mounting plate is provided with a first waist-shaped hole extending in the front-to-back direction, and the binocular camera mounting seat is fixed by a first bolt passing through the first waist-shaped hole and cooperating with the first fixing screw hole;

[0015] A second fixing screw hole is provided on the back of the binocular camera, a second waist-shaped hole extending in the vertical direction is provided on the vertical mounting plate, and the binocular camera mounting seat is fixed by a second bolt passing through the second waist-shaped hole and cooperating with the second fixing screw hole.

[0016] The above setting provides greater flexibility and adjustability. Through the cooperation of waist-shaped holes and bolts, the position of the binocular camera can be easily fixed and adjusted while ensuring the firmness of the installation. This design provides flexibility and ensures stability during the acquisition process.

[0017] Preferably, each group of the slide rails is one, the two slide rails are linearly arranged in a row, and the two slide rails are connected together or there is a gap between them;

[0018] The mounting platform is also provided with a stopper, which is located between the two mounting seats and is used to limit the two mounting seats in a direction of approaching each other, thereby limiting the picking fingers from approaching too closely and preventing potential damage or interference.

[0019] Preferably, the two operating rods are located in the upper front of the handheld handle. When in use, the operator holds the handheld handle with one hand and operates the two operating rods to move, which improves the convenience and efficiency of operation and makes the data collection process smoother.

[0020] Preferably, the mounting platform in the present invention is also provided with a reset tendon or a reset spring for automatically resetting the two operating rods to positions away from each other, thereby reducing manual intervention and improving the continuity and efficiency of data collection.

[0021] Preferably, each of the mounting seats has a label setting plate extending upward, and the label setting plate is provided with an actuator opening positioning auxiliary label on the side facing the binocular camera.

[0022] The robot strawberry picking anthropomorphic operation data collection device of the present invention also includes a data processor, which is connected to the binocular camera via a data line and stores information collected by the binocular camera.

[0023] The present invention also provides a method for processing robot strawberry picking anthropomorphic operation data, using the robot strawberry picking anthropomorphic operation data collection device to collect strawberry picking anthropomorphic operation data, the collected data is processed to obtain the opening information and spatial posture information of the two picking fingers, plus the video information and IMU data information collected by the binocular camera, the four aspects of data information are all marked with corresponding timestamps to form the data required for strawberry picking robot training.

[0024] Beneficial effects of the present invention:

[0025] The use of the robot strawberry picking anthropomorphic operation data collection device and data processing method of the present invention can greatly improve the collection efficiency of the robot anthropomorphic data. Compared with the twin arms, the efficiency of single-person data collection is improved by more than ten times, and the collection design can be copied at low cost and distributed to multiple people for crowdsourcing collection data without the need for a robot entity. The real-time performance of the device also meets the requirements of real-time synchronous remote control robots. In general, the anthropomorphic operation data collection device and data processing method thereof have important efficiency improvement significance for constructing large-scale agricultural strawberry picking robot training data and obtaining embodied intelligent robots. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a front schematic diagram of the anthropomorphic operation data collection device for robot strawberry picking of the present invention;

[0027] Figure 2 This is a schematic diagram of a humanoid operation data collection device for strawberry picking robot according to the present invention, viewed from above;

[0028] Figure 3This is a schematic diagram of the humanoid operation data collection device for robot strawberry picking according to the present invention, viewed from below;

[0029] Figure 4 This is a schematic diagram of the back of the anthropomorphic operation data collection device for robot strawberry picking of the present invention;

[0030] Figure 5 This is a schematic diagram of the structure of the picking actuator of the present invention;

[0031] Figure 6 The left side schematic diagram of the picking actuator structure of the present invention;

[0032] Figure 7 This is a schematic diagram of the binocular camera mounting base;

[0033] Figure 8 This is a rendering of auxiliary labels for actuator opening positioning in the video taken by the binocular camera;

[0034] Fig. 9 Visualization of the trajectory of picking strawberries.

[0035] Markings in the figure: 1-installation platform, 11-first fixing screw hole, 12-stopper, 2-hand grip, 3-picking actuator structure, 31-picking finger, 32-mounting seat, 321-label setting plate, 33-slide rail, 34-slider, 35-operating lever, 36-actuator opening positioning auxiliary label, 4-binocular camera, 41-binocular camera mounting seat, 411-horizontal mounting plate, 4111-first waist-shaped hole, 412-vertical mounting plate, 42-second fixing screw hole. DETAILED DESCRIPTION

[0036] Example 1

[0037] Depend on Figure 1-7 As shown, the present invention provides a robot strawberry picking anthropomorphic operation data collection device, comprising: a mounting platform 1; a handheld handle 2, arranged below the mounting platform 1; a picking actuator structure 3, arranged at the upper front end of the mounting platform 1, comprising a pair of picking fingers 31 extending out of the mounting platform 1, each picking finger 31 is slidably connected to the mounting platform 1 through a mounting seat 32, and two groups of slide rails 33 are provided on the mounting platform 1, and two mounting seats 32 are respectively provided with sliders 34 that cooperate with one group of slide rails 33; each of the two mounting seats 32 is provided with an operating rod 35 extending downward through the mounting platform 1; each mounting seat 32 is also provided with an actuator opening positioning auxiliary label 36; a binocular camera 4, arranged on the mounting platform 1, and the field of view angle of the binocular camera 4 can see the picking fingers 31 and the actuator opening positioning auxiliary label 36; the binocular camera 4 has a built-in IMU for collecting IMU data;

[0038] When in use, the operator holds the handheld handle 2 and controls the movement of the two picking fingers 31 through the operating rod 35, and then the binocular camera 4 collects the opening and spatial posture information of the picking fingers 31.

[0039] The device of the present invention is equipped with a spatial attitude sensor: 1. binocular camera; 2. IMU. The IMU is built inside the binocular camera, so the primary data collected is binocular video and IMU data, and the secondary information obtained after processing is: 1. opening information; 2. spatial posture information.

[0040] When manually operating data collection, the picking fingers are operated by operating levers. The picking fingers facilitate the actuator to push aside obstructions and reach into the gap to clamp the strawberries and shorten the roots through a blade on one side (a blade is set above any one of the two picking fingers) (while keeping the clamped strawberries from falling).

[0041] In the present invention, the spatial attitude positioning binocular camera installation structure installs the binocular camera at an angle where the field of view angle can just see the actuator and the actuator opening positioning auxiliary label but cannot see other parts, ensuring that the video data collected by the system is exactly the same as the perspective when the robot actually uses the electric end effector to operate, and cannot be distinguished from the picture. Two positioning labels are pasted at the actuator opening positioning auxiliary label and the distance between the labels is calculated through algorithm processing to obtain the actuator opening information, so there is no need to install an opening sensor on the unpowered end effector, which further reduces costs and does not require complex circuit power supply.

[0042] In the present invention, in order to ensure that the camera has a better field of view to capture the dynamic changes of the picking finger and the actuator opening positioning auxiliary label, the binocular camera 4 is set at the upper rear end of the mounting platform 1 ( Figure 1-4 The binocular camera 4 can be arranged at the upper left rear end, the rear end or the right rear end of the mounting platform 1, as long as the blind area of ​​the visual field can be reduced and the accuracy and completeness of data collection can be improved.

[0043] Further, in order to provide greater flexibility and adjustability, the binocular camera 4 is mounted on the mounting platform 1 through a binocular camera mounting seat 41, and the binocular camera mounting seat 41 includes a horizontal mounting plate 411 and a vertical mounting plate 412;

[0044] The mounting platform 1 is provided with a first fixing screw hole 11, the horizontal mounting plate 411 is provided with a first waist-shaped hole 4111 extending in the front-to-back direction, and the binocular camera mounting seat 41 is fixed by a first bolt passing through the first waist-shaped hole 4111 and cooperating with the first fixing screw hole 11; the back of the binocular camera 4 is provided with a second fixing screw hole, and the vertical mounting plate 412 is provided with a second waist-shaped hole 4121 extending in the vertical direction, and the binocular camera mounting seat 41 is fixed by a second bolt passing through the second waist-shaped hole 4121 and cooperating with the second fixing screw hole. Through the first waist-shaped hole and the second waist-shaped hole, the position and angle of the camera can be easily adjusted to meet different acquisition requirements. In addition, the cooperation of the waist-shaped hole and the bolt can also conveniently fix and adjust the position of the binocular camera while ensuring the firmness of the installation.

[0045] In the present invention, each group of slide rails 33 is one, and the two slide rails 33 are linearly arranged in a row, and the two slide rails 33 are connected together or there is a gap between the two, which helps the picking fingers to move smoothly along a fixed direction; in order to prevent the picking fingers from getting too close and being damaged or interfered, a stopper 12 is also provided on the mounting platform 1. The stopper 12 is located between the two mounting seats 32 and is used to limit the two mounting seats 32 in the direction of approaching each other, thereby improving the safety and stability of the equipment.

[0046] In the present invention, two operating rods 35 are located at the upper front of the hand-held handle 2. When in use, the operator can easily hold the hand-held handle 2 with one hand and operate the two operating rods 2 to move, thereby controlling the movement of the two picking fingers. This design improves the convenience and efficiency of operation and makes the data collection process smoother.

[0047] In order to reduce manual intervention, the present invention is also provided with a reset tendon or a reset spring on the installation platform 1 for automatically resetting the two operating rods 2 to positions away from each other, or other tools that can automatically reset the two operating rods 2 to positions away from each other are used, such as the combination of an electromagnet and an iron material, etc., as long as the continuity and efficiency of data collection can be improved.

[0048] Each mounting seat 32 has a label setting plate 321 extending upward, and an actuator opening positioning auxiliary label 36 is provided on the label setting plate 321 facing the binocular camera 4. The actuator opening positioning auxiliary label on the label setting plate can accurately record the opening information of the picking finger. Extending upward can make the actuator opening positioning auxiliary label and the picking finger always in the same field of view, ensuring the accuracy of data collection.

[0049] The robot strawberry picking anthropomorphic operation data collection device of the present invention also includes a data processor, which is connected to the binocular camera via a data line and stores information collected by the binocular camera.

[0050] The present invention uses a self-assembled set of computing and operating systems to connect the binocular camera above the holding device to collect and record operation data. The core of the computing system is the Raspberry Pi 5 single-chip microcomputer, which runs the Raspbian operating system and deploys the camera driver, the display driver, and the interactive button driver through Docker. The computing system is small and can be held in one hand to facilitate single-person use by the data collector. The system consumes about 2-3W of energy, and the built-in battery can be used for continuous recording for 2-3 hours. The recording button is used to separate the operation data fragments. The self-assembled set of computing and operating systems is a prior art, which only temporarily stores the collected data here, and then exports the data for processing for the subsequent strawberry picking robot training.

[0051] When the collection object and the collector are the same, the twin arms in the prior art and the device of the present invention are used to collect data. The results show that the data collection speed of the twin arms is 2 minutes / data segment on average (including the time for initialization and reset, of which the valid data is about 8-10 seconds). When the data is collected by the device, the data collection speed is about 11-12 seconds / data segment on average, and basically all the data is valid. This shows that the use of the robot strawberry picking anthropomorphic operation data collection device of the present invention can greatly improve the collection efficiency of the robot anthropomorphic data.

[0052] Example 2

[0053] The present invention also provides a method for processing robot strawberry picking anthropomorphic operation data, using the robot strawberry picking anthropomorphic operation data collection device provided in Example 1 to collect strawberry picking anthropomorphic operation data, and processing the collected data to obtain the opening information and spatial posture information of the two picking fingers, plus the video information and IMU data information collected by the binocular camera, and the four aspects of data information are all marked with corresponding timestamps to form the data required for strawberry picking robot training.

[0054] The data required for training the strawberry picking robot of the present invention are divided into four aspects: 1. opening information, 2. spatial posture information, 3. video information collected by a binocular camera, and 4. IMU data information; the above four aspects of data are divided into two categories, among which 3 and 4 are primary data, 3 is directly output by the binocular camera, and 4 is directly output by the IMU in the camera.

[0055] A very important dimension in the robot operation data is: when and at what opening the robot operates the end effector, so the data acquisition system must provide this information. The end effector of the whole system of the present invention is without driving force. And in order to ensure the low cost and high reliability of the whole system and the requirement of no complex circuit, the picking finger does not have an opening sensor. By adding an actuator opening positioning auxiliary label in the front camera angle of the device, the opening information is obtained through the visual algorithm.

[0056] Among them, the method for obtaining the opening information is to process the binocular camera video information (the present invention only uses one of the cameras), identify the April Tag, and correct it to obtain the opening. Specifically, two April Tags with different IDs are installed on the device. The April Tag is a visual mark, just like a QR code. It consists of black and white squares, and each tag has a unique ID. By using the camera to recognize these tags, their position and posture in space can be obtained. It is often used in robot navigation, AR / VR, camera calibration and other fields to help machines understand the surrounding environment. Installing a tag here allows the video captured by the camera to see the tag (such as Figure 8 As shown in the figure, the information installed at the bottom of the camera's viewing angle is selected using the known tag, and irrelevant upper image information is removed to reduce the computational complexity of the next step of algorithm recognition. The specific position of the tag in the image is then obtained through the recognition algorithm in the AprilTag system, and the opening of the picking finger can be obtained through the correction program. The correction program automatically detects the maximum and minimum tag openings through a series of operation data, performs data filtering to remove unreliable extreme values, and obtains TO max (the maximum pixel distance of the tag on the image) and TO min (the minimum pixel distance of the tag on the image), thereby calculating the actuator opening EO (0%-100%).

[0057]

[0058] Where TO is the pixel distance of the current label on the image, in pixels.

[0059] For spatial pose information, the ORB-SLAM3 algorithm is used to simultaneously process the video stream of the binocular camera and the inertial measurement unit (IMU) data to achieve precise positioning of the camera system in three-dimensional space. Specifically, the spatial pose information is obtained by simultaneously inputting the video captured by the binocular camera and the IMU information into the ORB-SLAM3 algorithm. The function of the ORB-SLAM3 algorithm is to obtain the spatial pose of the camera movement through the video and IMU information of the binocular camera; in order to ensure the accuracy of the algorithm calculation and not be affected by the end effector and label in the camera's perspective, the visual features of the bottom part of the picture are filtered out. The bottom part refers to the part where the label and the end effector are located. The system can track the position and posture of the camera in real time, and generate detailed trajectory information. The visual trajectory information is projected onto the current image, such as Fig. 9 shown.

[0060] Specifically, by fusing the depth information of binocular vision and the high-frequency measurement data of IMU, namely: 1. By pre-integrating the IMU data, the displacement and rotation estimation of the camera in a short period of time is obtained, thereby supplementing and correcting the errors caused by low quality of depth images or rapid motion;

[0061] 2. Through the (Bundle Adjustment) method, the feature points of the depth image are taken into account together with the motion information provided by the IMU to optimize the camera trajectory and map construction. This optimization method can reduce the errors that may be caused by relying solely on depth images or IMU information, and provide more accurate and stable estimates.

[0062] The present invention significantly improves the robustness and positioning accuracy of the system, especially in scenes with sparse feature points or fast motion. The processed spatial trajectory information is synchronized with the camera's view image stream information and the end effector's opening information, so the data can be used to train the future end motion and actuator opening and closing of the robot arm when the corresponding image is observed.

Claims

1. A robot strawberry picking anthropomorphic operation data collection device, characterized in that: include: Installation platform; A hand-held handle is provided below the mounting platform; The picking actuator structure is arranged at the upper front end of the mounting platform, and includes a pair of picking fingers extending out of the mounting platform, each picking finger is slidably connected to the mounting platform through a mounting seat, and two sets of slide rails are arranged on the mounting platform, and two mounting seats are respectively provided with a slider that cooperates with one set of slide rails; each of the two mounting seats is provided with an operating rod extending downward through the mounting platform; each mounting seat is also provided with an actuator opening positioning auxiliary label; A binocular camera is arranged on the mounting platform, and the field of view of the binocular camera can see the picking finger and the actuator opening positioning auxiliary label; the binocular camera has a built-in inertial measurement unit for collecting inertial measurement unit data; When in use, the operator holds the hand-held handle and controls the movement of two picking fingers through the operating rod, and then the binocular camera collects the opening and spatial posture information of the picking fingers.

2. According to claim 1, the robot strawberry picking anthropomorphic operation data collection device is characterized in that: The binocular camera is arranged at the upper rear end of the installation platform.

3. According to claim 2, the robot strawberry picking anthropomorphic operation data collection device is characterized in that: The binocular camera is installed on the mounting platform through a binocular camera mounting seat, and the binocular camera mounting seat includes a horizontal mounting plate and a vertical mounting plate; The mounting platform is provided with a first fixing screw hole, the horizontal mounting plate is provided with a first waist-shaped hole extending in the front-to-back direction, and the binocular camera mounting seat is fixed by a first bolt passing through the first waist-shaped hole and cooperating with the first fixing screw hole; A second fixing screw hole is provided on the back of the binocular camera, a second waist-shaped hole extending in the vertical direction is provided on the vertical mounting plate, and the binocular camera mounting seat is fixed by a second bolt passing through the second waist-shaped hole and cooperating with the second fixing screw hole.

4. The robot strawberry picking anthropomorphic operation data collection device according to claim 1, characterized in that: Each group of the slide rails is one, the two slide rails are linearly arranged in a row, and the two slide rails are connected together or there is a gap between them; The mounting platform is also provided with a stopper, which is located between the two mounting seats and is used to limit the two mounting seats in a direction of approaching each other.

5. The robot strawberry picking anthropomorphic operation data collection device according to claim 1, characterized in that: The two operating rods are located in the upper front of the hand-held handle. When in use, the operator holds the hand-held handle with one hand and operates the two operating rods to move.

6. The robot strawberry picking anthropomorphic operation data collection device according to claim 5, characterized in that: The mounting platform is also provided with a reset tendon or a reset spring for automatically resetting the two operating rods to positions away from each other.

7. The robot strawberry picking anthropomorphic operation data collection device according to claim 1, characterized in that: Each of the mounting seats is provided with a label setting plate extending upward, and the label setting plate is provided with an actuator opening positioning auxiliary label on the side facing the binocular camera.

8. The robot strawberry picking anthropomorphic operation data collection device according to claim 1, characterized in that: It also includes a data processor, which is connected to the binocular camera via a data line and stores information collected by the binocular camera.

9. A method for processing data of anthropomorphic operation of robot strawberry picking, characterized in that: The robot strawberry picking anthropomorphic operation data collection device described in any one of claims 1 to 8 is used to collect strawberry picking anthropomorphic operation data, and the collected data is processed to obtain the opening information and spatial posture information of the two picking fingers, and the video information and IMU data information collected by the binocular camera are added. The four aspects of data information are all marked with corresponding timestamps to form the data required for strawberry picking robot training.

Images